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ABSTRACT 


This  thesis  addresses  the  instructional  gap  existent  between  the  theoretical  instruction  that 
is  typically  delivered  in  the  classroom  and  the  hands-on  training  in  periscope  simulators. 
In  those  navies  that  adopt  the  “eyes-only”  or  “perisher”  technique  for  training  of 
periscope  depth  safety  rules,  the  submarine  officers  need  to  gain  knowledge  and  master  a 
series  of  skills  and  abilities  prior  to  the  training  sessions  in  periscope  simulators.  This 
work  suggests  and  explores  the  use  of  web-based  simulation  as  a  tool  to  diminish  this 
gap,  applying  the  concept  of  part-task  training  to  enable  the  delivery  of  better-prepared 
officers  to  the  simulator  phase  of  training  and  better  leverage  the  time  spent  at  a  full-scale 
simulator.  To  test  this  concept,  a  prototype  web-based  periscope  simulator  was 
developed,  and  a  usability  study  was  conducted  among  the  body  of  students  of  the  Naval 
Postgraduate  School.  The  findings  revealed  that  most  of  the  participants  were  receptive  to 
web-based  simulation  for  training,  and  that  it  is  a  viable  and  promising  field,  but  some 
technical  caveats  were  identified.  Although  this  thesis  focused  on  the  periscope 
simulation,  the  concept  can  be  extended  and  applied  for  other  domains  in  which  a  similar 
gap  can  be  found  between  theory  and  hands-on  training. 
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I. 


INTRODUCTION 


A.  RESEARCH  PROBLEM  AND  MOTIVATION 

The  training  of  a  submarine  offieer  (SO)  is  a  long,  eostly,  and  continuous  process. 
Due  to  the  complexity  and  danger  involved  in  submarine  operations,  there  is  a  set  of 
basic  knowledge,  skills,  and  abilities  (KSA)  that  must  be  developed  by  the  officers  to  a 
minimum  acceptable  standard  before  they  can  be  assigned  to  any  role  in  the  operational 
environment.  Most  of  the  topics  that  need  to  be  learned  by  a  trainee  involve  a  proper 
utilization  of  the  equipment  and  resources  of  the  submarine.  This  fact  would  intuitively 
elect  the  submarine  itself  as  the  best  learning  tool.  Indeed,  in  most  navies,  a  considerable 
part  of  the  training  is  performed  aboard  an  actual  submarine.  However,  there  are  some 
drawbacks  that  make  it  very  difficult  or  even  unfeasible  to  conduct  courses  relying  on  the 
use  of  actual  equipment.  Here  are  several  factors  that  need  to  be  considered  before 
deciding  on  where  and  how  to  conduct  that  training: 

•  Space:  The  space  aboard  a  submarine  is  scarce,  and  generally  it 
accommodates  only  the  room  necessary  for  the  crew  to  live  and  work 
amid  equipment,  especially  in  conventional  submarines.  Generally,  each 
individual,  who  is  predominantly  the  trainee  in  a  deployment  condition, 
represents  one  fewer  bunk,  decreasing  the  comfort  of  the  crew.  Very  often 
it  is  just  not  feasible  to  have  an  entire  cohort  aboard,  forcing  more  than 
one  submarine  to  be  deployed  as  learning  tool  for  the  phases  of  practice  at 
sea.  Even  if  the  training  can  be  conducted  with  the  submarine  ashore,  the 
lack  of  space  can  represent  an  inconvenience  for  both  the  trainees  and  the 
crew. 

•  Stress  of  the  equipment:  The  SO  trainees  must  be  prepared  for  the  harshest 
possible  situations;  however,  although  the  submarine  equipment  is  made 
to  eventually  support  extreme  conditions  of  operation,  using  the  same 
equipment  repeatedly  in  those  conditions  for  training  purposes  could 
reduce  the  life  cycle  of,  cause  costly  damage  to,  and  reduce  the  reliability 
of  the  equipment. 

•  Costs:  The  costs  involved  in  training  conducted  aboard  will  not  be  limited 
to  those  related  to  the  equipment  maintenance  and  eventual  repair.  Since 
the  training  requires  the  submarine  to  be  at  sea,  considering  the  case  in 
which  the  submarine  is  assigned  only  for  training,  the  cost  can  grow 
sharply.  That  cost  will  involve  the  costs  to  maintain  the  whole  crew,  fuel, 
etc.  These  operational  costs  are  multiplied  to  each  unit  at  sea,  and  they 
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grow  even  larger  when  the  exercise  requires  the  presence  of  surface  ships, 
aircrafts,  and  other  submarine  units,  as  is  the  case  for  periscope  tactics 
training. 

•  Danger.  Underwater  operations  always  represent  high  risks.  Although  the 
trainees  generally  are  “shadowed”  by  experienced  instructors,  their 
instruction  will  often  require  the  submarine  to  be  put  in  dangerous 
situations  where  any  flaw  can  be  hard  to  control  and  reverse. 

These  drawbacks  are  considerably  minimized  if  not  completely  removed  by  using 
simulators  to  teach  the  trainees  before  sending  them  to  a  final  practical  phase  aboard  the 
submarine.  In  Brazilian  Navy  Submarine  School  (BNSS),  these  simulators,  called  Attack 
Trainers  (AT),  are  used  to  teach  and  train  tactical  maneuvers.  They  are  built  as  full  mock- 
ups  of  the  submarine  Command  Room  (henceforth  referred  to  as  “Control  Room”),  with 
the  main  sensors  and  equipment  reproduced  on  a  one-to-one  scale.  During  the  course  of 
their  career,  Brazilian  Navy  SOs  are  submitted  to  at  least  three  main  courses  in  which 
they  use  those  AT  to  build  the  KSA  necessary  to  participate  in  a  Submarine  Command 
Course  (SMCC):  Basic  Submarine  Specialization  Course,  Preparation  for  the  Duty  of 
Periscope  and  Sound,  and  Submarine  Operations  for  Officers.  All  these  courses  are 
developed  in  three  different  environments:  classroom,  AT,  and  aboard  a  submarine  at  the 
sea. 

In  the  Basic  Submarine  Specialization  Course,  the  students  have  their  first  contact 
with  the  subject  of  submarine  safety  and  submarine  operations,  but  the  focus  is  mostly  in 
theory.  The  newly  graduated  SO  is  not  qualified  for  service  in  the  Control  Room  as  the 
Officer  of  the  Deck  (OOD).  Although  they  have  this  first  contact  with  theory,  SOs  do  not 
practice  in  the  tactical  procedures,  and  have  contact  with  the  AT  only  as  observers.  The 
practical  phase  aboard  is  focused  on  the  operation  of  equipment  in  preparation  for  the 
qualifying  test. 

In  the  next  course,  the  Preparation  for  the  Duty  of  Periscope  and  Sound,  SOs  are 
prepared  to  be  qualified  as  OODs.  This  is  generally  the  first  opportunity  in  which  they 
will  use  the  AT.  In  this  course  the  students  need  to  apply  the  knowledge  they  obtained  in 
the  previous  course.  At  the  end  of  the  course  the  students  have  a  phase  at  sea,  but  with  the 
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submarine  operating  alone;  the  focus  is  on  the  maintenance  of  safety,  not  in  submarine 
warfare  (SW)  actions. 

The  third  course,  Submarine  Operations  for  Officers,  aims  to  expand  the  KSA 
acquired  during  the  second  course,  focusing  on  tactical  operations  in  SW.  For  the  sea 
phase  the  trainees  are  required  to  plan,  design,  and  execute  SW  operations  in  conditions 
similar  to  wartime  scenarios.  Besides  the  participation  of  a  submarine,  anti-submarine 
warfare  (ASW)  ships  and  aircrafts  are  mobilized  for  the  sea  phase  in  this  course. 

After  those  three  courses,  generally  at  the  rank  equivalent  to  Lieutenant- 
Commander  (04)  or  Commander  (05),  SOs  are  enrolled  in  an  SMCC.  Most  officers  do 
the  SMCC  at  BNSS,  and  some  are  sent  to  complete  this  course  in  other  allied  navies. 
Different  from  the  previous  courses,  the  SMCC  does  not  have  the  focus  on  teaching  new 
topics.  The  classroom  phase  is  treated  as  a  refresher  for  the  entire  content  learned  during 
the  whole  career. 

This  series  of  courses  and  education  is  spread  over  nearly  fifteen  years  of  an  SO’s 
career.  Each  course  is  a  pre-requisite  to  the  next  one,  but  they  can  be  separated  by  a  time 
frame  of  several  years.  In  the  intervals  between  courses,  each  officer  is  responsible  for 
keeping  up  the  KSA  previously  acquired.  Those  assigned  to  the  unit  next  to  BNSS  can 
ask  for  extra  training  at  the  AT;  those  training  opportunities  will  be  subject  to  the  tight 
schedule  of  regular  courses  and  maintenance.  The  officers  serving  in  units  far  from  the 
BNSS  will  have  to  find  another  way  to  prepare,  but  never  with  the  help  of  a  similar  three- 
dimensional  (3D)  simulation. 

Most  of  the  KSA  needed  to  match  the  exigency  of  the  practical  phases  in  the 
courses  require  lengthy  practice,  generally  more  than  the  timeframe  allotted  for  the 
course  itself.  Despite  the  advantages  brought  by  the  use  of  simulators,  they  are  not  always 
available  for  those  in  need  of  training,  for  reasons  such  as  schedule  or  geographic 
limitation.  Even  when  simulators  are  available  for  extra  training,  extending  the  hours  of 
their  use  could  mean  an  increase  of  the  costs  of  operation.  A  web-based  learning 
environment  (WBLE)  would  help  minimize  those  drawbacks  by  offering  an  always- 
available  tool  for  training,  which  users  could  adapt  to  their  own  schedule. 
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The  criteria  considered  to  appoint  Commanding  Officers  (COs)  of  ships  are 
similar  in  most  navies.  However,  unlike  on  the  surface  ships,  to  be  the  CO  of  a 
submarine,  an  SO  has  to  satisfy  the  common  criteria  and  he  also  has  to  pass  through  the 
last  test  to  compete  for  a  command  post:  it  is  necessary  to  be  approved  in  the  SMCC. 
This  last  step  involves  a  course  that  takes  about  three  to  four  months,  with  small  cohorts 
of  up  to  six  candidates.  The  exact  format  may  vary  from  one  navy  to  another,  but  it  is 
very  often  a  course  with  a  high  rate  of  failure.  Due  to  those  high  rates  of  failure,  the 
SMCC  is  commonly  known  as  “The  Perisher.”  There  is  only  one  chance  for  the  future 
submarine  Commanding  Officers  to  perform  this  course,  thus  it  is  the  ultimate  test  in  the 
officer’s  career  as  a  submariner.  Failing  SMCC  means  that  the  officer  will  never  serve 
aboard  the  submarine. 

All  those  characteristics  can  turn  the  SMCC  into  a  daunting  prospect  for  a  long 
time;  all  prospective  COs  know  they  must  be  prepared  when  the  time  comes,  for  they  will 
have  only  one  chance.  Although  the  theoretical  knowledge  and  tactical  skills  are 
necessary  to  perform  well  in  this  course,  the  main  objective  of  SMCC  is  not  to  teach 
those  subjects,  but  to  test  if  the  candidates  are  able  to  keep  acting  as  expected  under  a 
warlike  situation,  when  they  are  pressed  by  time  and  the  threats  are  presented  to  them. 
The  tools  to  succeed  in  SMCC  are  built  during  the  officer’s  career,  whether  with  other 
courses  or  with  the  daily  hands-on  practice  while  on  duty  at  the  sea.  The  courses 
generally  have  phases  of  traditional  classroom  lectures,  simulator  practice,  and  final  tests 
at  the  sea.  The  simulator  is  where  SOs  have  the  first  opportunity  to  connect  the 
knowledge  and  skills  from  the  various  sources.  Some  set  of  KSA  necessary  to  begin 
training  at  the  simulator  cannot  be  developed  during  class  time — they  rely  on  massive 
amounts  of  information  that  needs  to  be  acquired  through  a  self-study.  While  the  theory 
is  taught  in  the  classroom,  each  student  still  must  find  additional  time  to  memorize  the 
techniques  and  practice  the  rapid  mental  calculations  (RMC)  that  they  will  be  required  to 
perform  in  seconds  at  the  simulator  training  session.  Once  the  students  arrive  at  the 
simulator,  they  are  expected  to  have  practiced  those  KSA  to  a  minimum  acceptable  level 
of  proficiency;  otherwise,  it  will  be  very  difficult,  sometimes  impossible,  to  execute  the 
simulator  missions. 
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A  shared  understanding  among  those  who  have  participated  in  such  courses  is  that 
the  trainee  who  masters  the  basic  skills  is  not  guaranteed  to  perform  well,  but  the  one 
who  fails  to  master  them,  generally  has  a  very  bad  overall  performance,  or  may  even  fail 
the  courses.  The  causes  for  bad  preparation  vary.  Some  trainees  arrive  without  proper 
preparation  because  their  current  assignment  did  not  provide  them  with  enough  spare 
time  to  practice.  Others  begin  the  courses  unprepared  because  they  did  not  have  the  right 
guidance,  or  the  method  that  they  had  created  to  practice  was  not  efficient,  even  though 
they  worked  very  hard.  Perhaps  an  even  worse  case  is  the  students  who  do  not  fail  the 
course,  but  who  learned  concepts  by  heart  without  really  understanding  them.  After  some 
time,  the  long  list  of  memorized  rules,  if  those  did  not  include  critical  thinking  and  deep 
analysis,  will  be  more  difficult  to  recall,  and  the  officer  will  gradually  return  to  lower 
levels  of  readiness.  Being  that  the  objective  of  this  kind  of  training  is  to  improve  the 
efficiency  of  the  manpower,  such  deterioration  of  level  of  readiness  only  guarantees  that 
this  objective  will  not  be  achieved. 

Following  the  evolution  and  widespread  use  of  personal  computers  and  the 
Internet,  the  use  of  distance  learning  WBLE  is  expanding  nowadays.  They  arose  as  good 
solutions  for  professionals  with  a  tight  work  schedule  who  are  unable  to  invest  time  in 
more  traditional  forms  of  education.  Driven  by  these  facts,  this  thesis  suggests  the  use  of 
web-based  simulation  as  a  tool  to  be  offered  to  SOs,  to  fill  this  gap  between  classroom 
lectures  and  simulator,  taking  advantage  of  the  possibility  to  deliver  3D  simulations  using 
the  web  browsers  and  web  technology.  The  objective  is  to  create  an  effective  and 
efficient  WBLE  that  will  be  used  by  the  SOs  to  guide  and  optimize  their  preparation  time. 
The  WBLE  will  illustrate  very  similar  conditions  and  scenarios  the  students  will  be 
exposed  to  in  the  simulator  when  they  take  the  training.  The  benefit  of  this  approach  is  to 
allow  trainees  to  form  a  solid  basis  before  they  begin  the  courses.  The  expectation  is  that 
students,  once  they  have  practiced  different  procedures  and  scenarios  in  WBLE,  will  be 
able  to  take  the  best  of  their  simulator  training  session  and  focus  their  efforts  on  the 
development  of  more  advanced  skills,  such  as  situational  awareness  and  decision  making. 
The  web-based  simulation  could  also  help  those  SOs  serving  abroad,  or  the  SOs  who 
spend  long  periods  of  time  outside  the  submarine;  both  groups  need  to  remind  themselves 
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of  the  concepts  and  recall  their  skills  in  preparation  for  the  return  to  the  submarine  or  for 
enrollment  in  an  SMCC. 

B.  RESEARCH  QUESTIONS 

The  issues  exposed  in  the  previous  sections  led  to  the  following  research 
questions: 

1.  What  are  the  advantages  of  using  a  web-based  3D  simulation? 

2.  What  are  the  main  knowledge,  skills,  and  abilities  (KSA)  necessary  for  the 
performance  of  the  Approach  Officer  (AO)  and  his  role  in  a  submarine, 
which  can  be  either  developed  or  improved  with  the  help  of  a  tactical 
periscope  simulator? 

3.  Which  of  those  KSA  could  be  improved  with  the  help  of  a  WBLE,  without 
the  direct  help  of  an  instructor  and  outside  the  classroom  environment? 

4.  Do  WebGL  compatible  web-browsers  have  the  resources  to  fully  support  a 
web-based  3D  simulation? 

5.  What  are  the  usability  issues  of  a  web-based  3D  simulation  (efficiency, 
effectiveness,  and  user  satisfaction  with  the  user  interface)? 

C.  SCOPE 

The  thesis  focuses  on  the  training  of  periscope  techniques  using  simulators;  a  goal 
is  to  identify  the  opportunities  for  delivering  training  using  web-based  periscope 
simulation.  This  type  of  training  would  represent  a  bridge  between  classroom  teaching 
and  training  in  the  full-scale  periscope  simulators. 

A  prototype  of  a  Web-based  Periscope  Simulation  developed  using  open-source 
web-based  technology  is  used  to  conduct  a  usability  study  and  identify  possible  issues  of 
this  type  of  technology  and  training  modality. 
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D.  TECHNICAL  APPROACH 


To  meet  the  proposed  objeetives,  the  following  four-step  approach  has  been 
adopted  for  this  thesis  work:  literature  review,  task  analysis,  prototype  development,  and 
usability  study. 

In  the  first  phase,  a  literature  review  will  be  conducted  about  the  use  of  3D 
simulations  to  teach  SO  and  about  the  characteristics  of  web-based  instruction.  The 
objective  is  to  gather  knowledge  available  in  literature  about  the  benefits,  issues,  and 
limitations  involved  in  the  combination  of  3D  simulations,  WBLE,  and  training, 
regarding  the  formation  of  SOs. 

To  expand  and  particularize  the  understanding  of  the  task  of  the  operator,  a  task 
analysis  is  conducted  about  the  tasks  and  training  sessions  performed  at  the  periscope 
simulator  in  BNSS.  The  main  objective  of  the  task  analysis  is  to  list  the  KSA  needed  for 
an  SO  to  perform  correctly  at  the  periscope.  In  addition,  task  analysis  is  used  to  establish 
performance  metrics  and  identify  tasks  suitable  to  be  implemented  and  trained  on  in  the 
WBLE.  Most  of  the  references  needed  to  perform  a  thorough  task  analysis  of  the  majority 
of  the  missions  performed  at  the  periscope  are  classified.  In  this  thesis,  only  one  mission 
has  its  task  analysis  reproduced,  providing  the  level  of  detail  that  can  be  found  in 
ostensive  publications. 

Based  on  the  task  analysis,  the  requirements  for  the  WBLE  prototype  are  defined. 
Observing  the  requirements  defined  in  the  previous  phase,  a  prototype  is  developed  to 
implement  those  tasks  in  a  web-based  environment.  The  main  technology  used  for 
development  is  WebGL,  an  open-source  JavaScript  API,  that  allows  rendering  3D 
graphics  within  Internet  browsers,  without  the  need  for  any  plug-in  or  installation  in  the 
user’s  computer. 

In  the  last  phase,  the  usability  of  the  prototype  is  tested;  test  subjects  were  able  to 
access  a  version  of  the  prototype  published  on  a  website  and  complete  a  survey  about 
their  experience.  The  usability  study  aims  to  assess  the  usability  of  the  human-computer 
interface  built,  identifying  the  points  of  strength  and  weaknesses,  collect  insight  about  the 
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feasibility  of  the  technology  for  wide  distribution,  and  gather  the  subjective  opinion  of 
the  average  Internet  user  about  this  modality  of  training. 


E.  THESIS  CONTRIBUTIONS 

The  main  contribution  of  this  thesis  is  a  series  of  understandings  related  to  a 
possible  application  of  the  web-based  3D  simulations  in  military  training.  The  data  about 
the  viability  and  utility  of  a  3D  web-based  simulation  for  training  was  gathered  and 
analyzed  for  more  detailed  insights.  The  results  of  the  survey  provided  insights  about  the 
user’s  online  habits  and  their  acceptance  of  web-based  training. 

The  study  also  collected  and  organized  a  set  of  understandings  about  the  cognitive 
processes  and  the  mindset  of  AOs  during  SW  operations,  particularly  for  the  operation  of 
diesel-electric  submarines  on  periscope  depth  (PD) — a  depth  in  which  it  is  possible  for  a 
submerged  submarine  to  raise  the  periscope  mast  over  the  sea  surface. 

The  WebGL  software  standard  used  to  develop  the  prototype  training  simulation, 
allows  3D  rendering  in  the  web-browser  without  the  need  for  any  software  installation. 
However,  the  WebGL  standard  is  currently  not  completely  supported  on  all  hardware  and 
software  platforms  available  on  the  market.  The  data  and  knowledge  we  collected  about 
this  standard  and  the  results  of  our  usability  study  allowed  us  to  highlight  the  advantages 
and  limitations  of  this  technology,  and  to  give  the  insights  about  the  feasibility  of  a  wide 
distribution  of  simulations  that  would  use  this  technology. 

F.  THESIS  STRUCTURE 

This  thesis  is  organized  as  follows: 

Chapter  I  states  the  problem  addressed,  presents  the  motivation  for  the  study, 
names  the  objectives  of  this  thesis,  and  describes  the  methodology  adopted  to  approach 
the  problem  and  reach  the  objectives  stated. 

Chapter  II  provides  the  background  information  necessary  to  better  understand  the 
problem  stated.  The  peculiarities  and  difficulties  of  SW  within  the  scope  of  this  thesis  are 
explained  in  more  detail,  discussing  the  cognitive  aspects  and  mental  models  for  decision 
making  inside  the  SW  context. 
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Chapter  III  consists  of  a  literature  review  divided  in  two  parts.  The  first  part 
presents  the  results  of  a  review  about  the  use  of  3D  simulations  for  submarine  training, 
discussing  the  suitability  and  efficiency  of  those  simulations  as  training  tools.  In  the 
second  part,  an  analysis  is  made  about  the  characteristics  of  WBLE  regarding  the 
advantages,  disadvantages,  and  peculiar  pedagogical  aspects  of  it. 

Chapter  IV  describes  the  missions  performed  at  BNSS  AT,  and  presents  the  task 
analysis  conducted  in  one  main  mission  to  identify  the  tasks  that  would  be  suitable  for 
implementation  in  a  WBLE.  The  learning  objectives  and  measures  of  performance 
(MOP)  are  then  identified  for  these  tasks  to  finally  list  the  requirements  for  designing  the 
WBLE. 

Chapter  V  outlines  how  the  Web-based  Periscope  Simulator  (WEBS)  was 
designed  and  prototyped. 

Chapter  VI  discusses  the  design,  execution,  and  results  of  a  usability  study  to  test 
the  WEBS  prototype. 


9 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


10 


II.  BACKGROUND 


The  Web-Based  Periscope  Simulation  (WEBS)  focuses  on  the  training  of  SOs  for 
techniques  to  be  used  at  PD;  this  chapter  briefly  introduces  some  concepts  behind  these 
techniques.  A  basic  comprehension  of  some  periscope  techniques  will  be  indispensable  to 
analyze  the  missions  executed  at  PD,  and  this  chapter  will  also  help  readers  understand 
the  cognitive  processes  related  to  the  scope  of  this  thesis. 

A.  THE  PERISCOPE  TECHNIQUES 

The  techniques  used  for  operation  of  the  periscope  were  developed  as  a  trade-off: 
the  goal  was  to  maximize  information  acquisition  with  the  least  possible  submarine 
indiscretion.  Some  of  these  techniques  began  to  be  developed  as  early  as  World  War  I, 
when  most  submarine  attacks  were  performed  within  the  visual  range,  in  a  very  close 
approach  to  surface  ships.  Modem  submarine  weapons  and  sensors  have  ranges  beyond 
the  horizon,  making  it  very  unlikely  that  a  submarine  will  get  as  close  to  surface  vessels 
(contacts)  as  was  typically  the  case  until  World  War  II.  However,  the  skills  necessary  to 
maneuver  a  submarine  safely  in  the  presence  of  several  surface  contacts,  using  only 
stopwatches  and  mental  agility,  remain  valuable  for  the  training  of  a  submarine  CO 
(White,  1993).  Some  periscope  techniques  are  still  applicable  nowadays,  either  because 
they  are  still  useful  in  the  modem  SW  scenario,  or  because  they  help  to  instill  in  the  SO 
the  mentality  of  safety  and  self-confidence.  Furthermore,  they  are  also  valuable  to  teach 
mental  models.  Even  though  nuclear  submarines  have  a  different  employment  doctrine, 
the  experience  with  courses  like  the  SMCC,  where  those  techniques  are  covered, 
represent  an  opportunity  to  observe  the  inside  of  diesel-electric  submarines  and 
understand  the  mindset  of  a  conventional  submarine  CO  (Steketee,  2004). 

The  periscope  is  the  only  sensor  capable  of  providing  a  complete  set  of  data  about 
the  tactical  situation  around  the  submarine  in  a  matter  of  a  few  seconds.  However,  as  an 
optical  sensor,  it  has  all  limitations  naturally  imposed  by  the  light  spectram.  The  range  of 
a  periscope  is  geometrically  limited  at  sea  due  to  the  Earth’s  curvature,  and  it  is 
susceptible  to  atmospheric  conditions  influencing  visibility.  Besides,  at  PD  the  submarine 
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is  made  more  vulnerable  to  counter-detection  by  ASW  surface  and  air  units.  However, 
despite  those  disadvantages,  periscopes  continue  to  be  of  paramount  importance  to  ensure 
the  safety  to  the  dived  submarine,  and  to  collect  or  confirm  tactical  information. 

Due  to  the  inherent  margin  of  errors  and  uncertainty  of  other  passive  sensors,  the 
periscope  is  the  only  sensor  able  to  resolve  with  conviction  the  potentially  questionable 
data  sets  acquired  by  other  sensors.  The  operator  is  able  to  collect  reliable  information 
about  the  surrounding  tactical  situation  in  a  matter  of  seconds,  calculate  the  geographical 
position  of  the  submarine  and  reveal  the  identity  of  targets.  The  information  collected  by 
a  periscope  usually  can  be  used  right  away,  because  they  are  useful  without  the  need  of 
further  processing  (other  than  human  interpretation). 

1.  Safe  Depth 

The  concept  of  safe  depth  (SD)  is  a  theoretical  base  that  drives  most  of  the  other 
safety  calculations  for  a  submarine  at  PD.  It  is  the  depth  at  which  the  submarine  can 
transit  within  a  safe  distance  from  the  keel  to  the  sea  bottom  while  still  allowing  ships  to 
safely  pass  over  the  fin  (see  Figure  1).  The  Standardization  Agency  for  North  Atlantic 
Treaty  Organization  (NATO)  defines  SD  in  a  following  way: 

A  submarine  is  said  to  be  at  Safe  Depth  when  its  keel  depth  is  such  as  to 
provide  the  required  separation  between  the  top  of  the  fixed  structure  of 
the  submarine  and  the  lowest  point  of  any  ship,  other  submarine  assigned 
to  a  higher  layer,  towed  ASW  device  and/or  helicopter  sonar  systems 
allowed  in  the  orders  for  the  exercise.  When  more  than  one  towed  device 
is  being  used  in  the  exercise.  Safe  Depth  applies  to  the  deepest  device 
being  employed.  (North  Atlantic  Treaty  Organization  [NATO],  2002) 


12 


LVSS 

_ y _ 

MAXIMUM  PERMISSIBLE  OPERATING  DEPTH 


Figure  1.  Safe  depth  calculations  (from  North  Atlantic  Treaty  Organization, 

2002,  pp.  2-13). 

Thus,  the  SD  will  depend  on  the  types  of  ships  expected  to  be  navigating  on  a 
given  Submarine  Patrol  Area  (SPA)  to  which  the  submarine  was  assigned  or  in  part  of  a 
transit  route. 

2.  “Go  Deep  Circle” 

For  a  given  SD  calculated,  each  submarine  class  will  need  a  certain  amount  of 
time  to  go  deep  according  to  its  rate  of  descent  (movement  from  PD  to  SD).  The  Go  Deep 
Circle  (GDC)  is  a  locus  of  the  distances  measured  from  the  submarine.  It  represents  the 
distance  that  can  be  covered  by  a  surface  ship  while  the  submarine  is  diving  from  PD  to 
SD.  In  Figure  2  there  is  an  example  of  a  GDC  calculation. 

In  periscope  drills  conducted  for  the  safety  phase  of  courses,  the  trainees  are 
required  to  do  RMC  to  figure  out  the  GDC  of  new  contacts.  Aiming  to  exemplify  the 
cognitive  load  imposed  on  the  mind  of  the  AO  by  RMC,  the  calculations  for  the  situation 
in  Figure  2  will  be  detailed:  the  submarine  in  this  example  takes  a  time  equal  to  60 
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seconds  to  go  deep,  travels  with  a  speed  of  seven  knots,  and  is  observing  the  Ship  A, 
which  develops  a  maximum  speed  of  29  knots.  GDC  calculation  always  considers  the 
worst-case  situation  to  build  a  buffer  of  safety.  For  the  given  example,  the  submarine  and 
the  Ship  A  are  on  the  same  line,  moving  toward  each  other.  The  relative  velocity  of 

approach  is  the  vector  sum  of  the  Ship  A  and  submarine  velocities. 


Figure  2.  Calculation  of  the  Go  Deep  Circle. 


r  Vo  100 

GDC\yards\  =  ^  180  T^j^\seconds\ 

(1) 

GDC  =  ^  6Q  ^  uooyards 

180 

(2) 

With  a  relative  velocity  of  36  knots,  the  submarine  and  the  ship  will  approach 
1,200  yards  in  60  seconds.  This  is  the  calculated  GDC  radius  for  this  class  of  ship.  If  any 
ship  with  the  same  characteristics  of  Ship  A  reaches  the  GDC,  the  submarine  will  be 
forced  to  go  deep. 

While  this  calculation  renders  more  safety,  it  extends  the  worst-case  to  the  whole 
circle,  giving  less  flexibility  to  the  submarine.  There  are  several  practical  ways  to 
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decrease  the  GDC  by  sectors,  aiming  to  confer  more  flexibility  to  the  submarine,  while 
still  keeping  it  safe.  The  mathematical  fundament  to  reduce  the  GDC  by  sectors  is 
exemplified  in  Figure  3.  However,  the  exact  way  it  is  applied  can  be  different  in  each 
navy,  and  is  generally  based  on  classified  publications. 


Figure  3.  GDC  reduction  by  sectors. 

In  this  example,  in  order  to  create  a  reduced  GDC  sector,  the  submarine  velocity 
vector  was  projected  45  degrees  to  each  side,  and  this  value  was  extended  to  the  rear 
sector,  as  shown  in  Equation  (4)  Ship  B  has  a  maximum  speed  of  25  knots. 

CflC.,,..,  =  60  =  1  myards  (3) 

3.  “All  Round  Look”  Interval 

The  All  Round  Look  (ARL)  is  a  basic  periscope  technique.  It  consists  of  turning 
the  periscope  all  the  way  around  to  quickly  obtain  situation  awareness  about  the 
environment  above  the  sea  surface.  The  periscope  mast  is  raised  to  execute  the  technique 
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and  lowered  just  after  completing  it.  The  whole  movement  must  be  executed  as  fast  as 
possible  to  minimize  the  exposure  of  the  periscope  mast,  but  slow  enough  to  make  sure 
that  all  visible  contacts  between  the  submarine  and  the  horizon  will  be  detected. 

To  calculate  the  acceptable  time  interval  between  two  ARL’s,  the  worst-case  is 
considered:  a  new  contact  appears  just  after  the  periscope  looks  toward  its  direction.  This 
contact  is  not  detected  and  approaches  the  submarine  with  its  maximum  speed  on  the 
closest  approach  distance.  The  time  it  takes  for  this  contact  to  reach  the  GDC  perimeter 
must  be  greater  than  the  maximum  interval  between  two  consecutive  ARLs,  in  order  to 
ensure  that  it  will  be  seen  before  reaching  the  GDC. 

If  more  than  one  type  of  contact  is  expected  for  a  given  SPA,  the  smallest  interval 
will  be  assumed  (shown  in  Table  1).  For  the  simplicity  the  example  one  can  consider  the 
submarine  traveling  at  seven  knots  and  having  the  same  SD  for  the  three  different  classes 
of  ships,  taking  exactly  one  minute  to  go  deep.  Therefore,  for  the  example  in  Table  1,  the 
ART  interval  to  be  considered  is  three  minutes  and  ten  seconds. 


Table  1.  Example  of  ARE  Interval  calculation. 


Type  of 
Contact 

Sighting 

Distance 

(yards) 

Maximum 

Speed 

(knots) 

Approximation 

Rate 

(yards/minute) 

GDC 

Radius 

(yards) 

Time 

to 

Reach 

GDC 

Ship  class  “A” 

5000 

29  knots 

1200 

1200 

3’10” 

Ship  class  “B” 

8000 

23 

1000 

1000 

T 

Ship  class  “C” 

2500 

8  knots 

500 

500 

4’ 

4.  “Look”  Intervals 

After  a  new  contact  is  detected,  periscope  observations  can  obtain  further  data, 
such  as  the  classification,  distance,  speed,  and  course  of  that  contact.  Knowing  the  exact 
distance  of  the  contact  and  a  point  in  time,  it  is  possible  to  calculate  how  much  time  this 
contact  would  take  to  reach  the  GDC  and  jeopardize  the  submarine  using  the  same 
reasoning  as  ARE  intervals.  This  calculated  amount  of  time  is  known  as  look  interval 
(LI).  For  each  new  contact  there  will  be  an  LI  to  be  controlled,  and  at  every  new 
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observation  this  LI  will  be  updated  accordingly.  One  of  the  abilities  required  from 
trainees  in  some  submarine  courses  is  to  be  able  to  mentally  calculate  and  control  several 
Lis  and  the  ARL  interval  at  the  same  time,  using  separate  stopwatches  (White,  1993). 

B.  COGNITIVE  ASPECTS  IN  PERISCOPE-DEPTH  OPERATIONS 

This  section  analyzes  the  main  aspects  of  AO  cognition  during  the  approach 
phase;  different  cognitive  theories  will  be  leveraged  to  provide  this  understanding.  A  full 
comprehension  of  the  cognitive  processes  that  occur  while  the  AO  is  addressing  an 
operational  situation  is  needed  to  understand  the  benefits  that  WBLE  adds  to  an 
individual’s  training. 

An  act  of  planning  the  approach  of  the  submarine  towards  an  objective  represents 
a  strenuous  cognitive  workload  for  the  AO.  This  is  especially  the  case  when  the  approach 
has  to  be  made  on  PD  and  in  shallow  waters.  The  AO  is  pressed  by  time  and  a  sense  of 
jeopardy.  The  decisions  must  be  made  relying  on  information  with  a  high  level  of 
uncertainty  aggregated.  The  situation  can  change  dynamically,  yet  the  decisions  must  be 
taken  upfront,  betting  on  predictions  of  future  situations.  The  objective  of  this  section  is 
to  analyze  the  existing  theories  about  the  cognitive  processes  of  an  AO  in  this 
environment  and  its  specific  conditions. 

1.  Naturalistic  Decision  Making 

The  characteristics  of  the  situations  in  which  the  AO  has  to  make  decisions  are: 
time  constraints,  high  uncertainty,  high  risk,  a  complex  and  dynamically  changing 
environment,  ill-defined  and  competing  goals  and  sub-goals.  All  of  this  makes  the 
domain  related  to  the  naturalistic  decision-making  (NDM)  theory  (Kirschenbaum,  2001). 
In  the  classic  decision-making  model,  it  is  assumed  that  the  decision  maker  generates  a 
list  of  courses  of  actions  and  then  chooses  the  best  solution  in  the  list  of  possible 
solutions.  While  doing  that,  the  decision  maker  is  conducting  a  thorough  analysis  of  all 
available  information  and  considering  all  hypotheses  under  the  cost-benefit  criterion. 
This  framework  of  decision  making  turns  out  to  be  unrealistic  for  emergent  situations  and 
harsh  environments  (Hutchins,  1996).  On  the  other  hand,  NDM  theory  rests  on  the  belief 
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that  the  decision  maker  passes  through  a  more  naturalistic  process  of  perception, 
comprehension  of  the  situation,  and  generation  of  appropriate  responses  (Klein,  2008). 

Amid  the  NDM  theories,  perhaps  the  best  known  and  best  fit  for  the  submarine 
problem,  is  the  recognition-primed  decision  (RPD).  The  RPD  theory  relies  on  two  key 
concepts:  situation  assessment  and  mental  simulation  (Klein,  1993).  In  RPD,  the  decision 
maker  assesses  the  situation  to  identify  a  satisfactory  course  of  action,  and  then  evaluates 
the  option  through  mental  simulation,  rather  than  making  cost-benefit  comparisons  on  all 
available  options  (Klein,  1993).  In  situation  assessment,  when  presented  with  a  new 
problem  that  needs  to  be  solved,  people  use  their  previous  experience  with  similar  past 
situations,  recognize  patterns  and  match  with  the  features  of  the  current  problem.  They 
automatically  recall  and  adapt  information  learned  from  the  previous  known  situation  to 
help  solve  the  current  problem.  When  they  are  faced  with  a  new  problem  for  which  they 
have  no  similar  experiences  in  the  past,  they  will  consciously  use  mental  simulation  to 
understand  the  problem  and  create  viable  solutions  (Shattuck  &  Miller,  2006). 

Analyzing  the  decision-making  process  of  a  submarine  approaching, 
Kirschenbaum  (2001)  found  that  the  AO  “uses  his  expertise  to  recognize  and  instantiate 
appropriate  schema.  He  uses  this  schema  as  the  basis  for  action  in  an  interactive  feedback 
loop”  (p.  203).  This  feedback  loop  is  based  on  two  mechanisms:  (1)  common 
information-gathering  strategies  and  (2)  shallow  goal  stack  that  allows  the  AO  to  rapidly 
adapt  to  changes  on  the  situation  (Kirschenbaum,  2001). 

2.  Situation  Awareness 

Situation  Awareness  (SA)  in  the  scope  of  SW  can  be  defined  as  the  understanding 
that  an  AO  has  about  the  current  tactical  situation  and  the  possible  developments  or 
consequences  in  a  close  future.  In  other  words,  in  order  to  support  the  process  of  decision 
making,  the  AO  builds  a  mental  model  in  his  or  her  mind.  Endsley  (1995)  proposes  the 
division  of  the  SA  process  into  three  major  hierarchical  phases: 
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1. 


Perception  of  pertinent  data  in  a  time  and  space  frame 


2.  Comprehension  of  the  perceived  data 

3.  Projection  of  a  near  future  situation 

In  the  context  of  PD  operations,  the  first  phase  represents  the  collection  of  data 
relevant  to  the  situation  by  the  SO.  These  data  are  about  the  own  submarine,  the 
environment,  and  the  other  units  detected.  Table  2  lists  some  of  the  most  common  data 
collected  by  the  AO  in  PD  operations.  As  noted  by  Kirschenbaum  (2001),  most 
traditional  models  consider  that  all  the  necessary  data  will  be  available  in  this  phase  to 
support  the  individual  in  building  his  or  her  SA.  However,  in  the  domain  of  SW,  each 
iteration  of  data  collection  is  likely  to  render  only  a  subset  of  the  necessary  data,  forcing 
the  AO  to  rely  on  estimations  and  suppositions  to  complete  the  necessary  operation.  Even 
the  data  successfully  obtained  can  be  misleading  due  to  the  large  amount  of  noise  added 
by  environmental  sources,  a  common  occurrence  for  information  from  the  sonar. 
Furthermore,  most  data  about  other  units  cannot  be  used  in  the  raw  form  of  collection.  To 
become  useful,  some  data  need  additional  mental  or  computational  processing.  For 
example,  to  obtain  the  course  of  a  surface  ship  using  the  periscope,  the  AO  collects  and 
inverts  the  ship’s  bearing,  visually  estimates  its  angle  on  the  bow  (AOB),  and  mentally 
calculates  the  course  as  the  sum  or  subtraction  of  these  two  values,  as  shown  in  Figure  4. 


Table  2.  Example  of  relevant  data  perceived  by  the  SO. 


Data  about  the  own  submarine 

Environmental  data 

Data  about  other  units 

-  Heading 

-  Course  over  ground 

-  Propeller  revolutions  per  minute 

-  Speed  over  ground 

-  Depth  of  the  keel 

-  Hull  inclination  (bubble) 

-  Visibility 

-  State  of  the  sea 

-  Direction  and  height 
of  the  sun 

-  Current 

-  Classification 

-  Bearing 

-  Distance 

-  Course 

-  Speed 

-  Level  of  danger 

-  Recent  data  history 

The  second  phase  consists  of  building  a  mental  model  of  the  current  situation;  the 
loose  elements  collected  previously  are  put  together  and  combined  according  to  existing 

patterns  learned  by  the  AO.  The  pieces  of  information  that  have  been  obtained  are 
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combined  to  gain  a  full  meaning,  converging  to  an  interpretation  of  what  is  possibly 
happening.  Unlike  some  other  domains,  a  domain  of  SW  is  characterized  by  the  existence 
of  a  high  degree  of  uncertainty.  Even  the  decisions  to  manoeuver  the  submarine  may 
affect  the  accuracy  of  data,  increasing  or  decreasing  the  level  of  uncertainty  (McKenna  & 
Little,  2000).  The  lack  of  data  and  the  errors  introduced  by  estimations  and  noise,  force 
the  AO  to  frequently  build  a  mental  model  of  the  situation  consisting  of  a  best  guess 
rather  than  a  complete  and  clear  comprehension  of  data.  It  is  similar  to  the  assembling  of 
a  jigsaw  puzzle  with  missing  pieces.  In  spite  of  the  uncertainty,  experienced  AOs  use 
their  mental  models  to  confirm  or  disregard  the  solutions  from  the  computer  and  the  data 
calculated  by  the  Submarine  Tracking  Party  (STP)  (Kirschenbaum,  Trafton,  Schunn  & 
Trickett,  2013). 


Figure  4.  Example  of  mental  calculation  of  a  contact’s  course,  based  on  visual 

estimation  of  the  angle  on  the  bow. 
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The  AO  derives  the  projection  of  the  future  status  using  his  understanding  of  the 
current  tactical  situation.  This  is  the  highest  level  of  SA  to  be  achieved  (Endsley,  1995,  p. 
37).  The  projection  into  the  future  situation  is  especially  important  for  conventional 
diesel-electric  submarines.  Due  to  the  speed  disadvantage  when  compared  to  most 
surface  vessels,  diesel-electric  submarines  must  make  decisions  and  act  in  advance, 
betting  on  a  prediction.  This  level  of  SA  must  be  achieved  in  a  timely  manner  if  the  goal 
is  to  make  correct  decisions  and  to  position  the  submarine  favorably  for  the  upcoming 
action.  Any  delay  or  miscalculation  can  cause  the  mission  to  go  irrecoverably  wrong. 
Particularly  complex  in  this  domain  is  the  fact  that  the  situation  is  dynamically  changing. 
Noise  and  dynamic  changes  in  attitude  are  deliberately  introduced  by  the  surface  units,  as 
they  try  to  deceive  the  submarine  solutions  (Ehret,  Benjamin,  Gray  &  Kirschenbaum, 
1997). 

The  human  perception  is  intertwined  with  the  system  sensing  of  the  environment 
throughout  the  process  of  SA  building;  the  AO  uses  information  processed  both  by 
humans  (the  own  AO  and  the  SAP)  and  by  machines.  A  good  model  for  cognitive 
processes  like  this  was  proposed  by  Shattuck  and  Miller  (2004),  as  illustrated  in  Figure  5. 
In  this  figure,  we  can  see  how  the  whole  set  of  data  from  the  environment  arrives  to  the 
AO  already  sieved  by  the  sensory  capacity  of  the  system  (represented  by  oval  2),  which 
limits  detection,  and  by  the  selective  displaying  of  detected  information  (oval  3),  as  some 
information  may  be  misjudged  or  considered  irrelevant  by  the  system  or  by  the  human 
operator. 
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Figure  5. 
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Data  perceived  by 
decision  maker 


Comprehension  of 
decision  maker 


Projection  of 
decision  maker 


©  Miller  and  Shattuck,  2003 


A  Dynamic  Model  of  Situated  Cognition  (from  Shattuck  &  Miller, 

2004,  p.  660). 


Once  the  information  reaches  the  AO,  it  will  be  subjected  to  what  the  model 
names  “lenses.”  Those  lenses  are,  in  this  context,  the  local  situation.  Operational  Order 
(OPORD),  doctrine  and  experience.  The  local  situation  will  lead  the  AO  to  pay  more  or 
less  attention  to  a  determined  set  of  information;  this  set  is  judged  to  be  more  important 
or  urgent  to  the  current  situation.  The  OPORD  represents  the  superior  orders,  the  mission 
assigned  to  the  submarine,  and  the  doctrine  is  a  general  set  of  guidelines  within  which  the 
AO  was  taught  to  act.  Finally,  the  experience  recalls  past  activities  that  the  AO  may  have 
practiced  before  that  bear  similarities  to  the  current  situation  (Shattuck  &  Miller,  2004). 

3.  The  Influence  of  Stress 

Stress  is  a  constant  factor  accompanying  submarine  operations,  and  it  is  essential 
to  understand  how  stressful  situations  can  influence  the  cognitive  process.  There  are 
many  sources  of  stress  for  the  AO:  the  characteristics  of  the  submarine  with  all  the  risks 

involved  in  operating  under  a  massive  body  of  water,  the  responsibility  for  making 
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decisions  that  may  have  harsh  consequences,  a  constant  uncertainty  of  information,  a 
time  pressure,  a  need  to  execute  several  parallel  tasks,  etc.  The  conditions  faced  by  the 
AO  match  those  listed  for  the  Tactical  Decision-Making  Under  Stress  (TADMUS) 
project,  conducted  by  the  United  States  Navy  (Cannon-Bowers,  Salas  &  Grossman, 
1992): 

•  Multiple  information  sources:  The  AO  must  manage  the  information  that 
he  or  she  collects  from  the  periscope,  the  information  available  from  the 
other  sensors  and  the  information  generated  by  the  STP. 

•  Incomplete,  conflicting  information:  Different  stations  of  the  STP 
frequently  diverge  about  the  solutions  found. 

•  Rapidly  changing,  evolving  scenarios:  The  opposing  forces  purposely 
insert  dynamic  changes  to  their  behavior,  to  difficult  the  action  of  the 
submarine. 

•  Requirement  for  team  coordination:  The  AO  is  the  conductor  of  the  STP. 

•  Adverse  physical  conditions:  An  approach  on  PD  can  be  very  physically 
demanding,  as  the  AO  must  successively  raise  and  lower  the  periscope. 

•  Performance  pressure:  The  AO  is  the  capital  decision  maker,  and  the 
responsibility  for  the  success  or  failure  of  the  mission  is  on  his  shoulders. 

•  Time  pressure:  An  AO  is  under  time  pressure  during  all  the  execution  of 
an  approach. 

•  High  work/information  load:  Besides  being  responsible  for  the  processing 
of  a  high  load  of  information  and  decision-making,  the  AO  still  has  to  care 
about  the  safety  of  the  submarine  and  the  aspects  concerning  the 
management  and  leadership  of  the  crew. 

•  Auditory  overload/interference:  Most  of  the  information  is  received 
verbally  by  the  AO. 

•  Threat  of  hostile  engagement:  Both  for  real  operations  and  for  exercises, 
where  the  surface  units  play  figurative  enemies. 
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One’s  concentration  and  selective  attention  can  be  severely  compromised  under 
high  stress,  impairing  the  individual’s  performance  and  significantly  affecting  the  ability 
to  perform  decision  making  (Entin  &  Serfaty,  1990).  Courses  like  the  SMCC  are 
designed  to  push  the  trainees  up  to  the  edge,  forcing  them  to  face  their  own  limitations, 
and  helping  them  to  improve  their  decision-making  ability  under  pressure  (Forster,  1996). 

One  of  the  approaches  for  the  training  of  decision  making  under  stress  is  part-task 
training  (PTT).  PTT  consists  of  decomposing  a  more  complex  task  into  components  that 
can  be  separately  trained  (Wightman  &  Lintern,  1985).  Kirlik,  Fisk,  Walker  and  Rothrock 
(1998)  list  some  lessons  learned  from  their  research  about  the  use  of  PTT  to  train 
dynamic  decision  making: 

•  To  train  high-level  cognitive  skills  and  decision-making,  ensure  that  the 
individuals  have  mastered  the  perceptual  and  motor  skills  that  they  will 
need  in  order  to  interact  with  the  environment. 

•  A  good  approach  to  train  these  basic  perceptual  and  motor  skills  is  PTT. 

•  The  components  of  an  overall  complex  task  when  identified  and  trained  to 
the  level  of  automaticity  help  to  alleviate  high- workload  situations. 

•  The  automation  of  components  of  a  task  turns  its  performance  more 
reliable  despite  of  the  insertion  of  stress  into  the  situation. 

•  PTT  must  be  supplemented  with  the  training  of  the  full-task. 

However,  Klein  (1996)  draws  attention  to  the  fact  that  stressors  can  either  disrupt 
or  improve  performance.  He  highlights  that  the  natural  adaptations  to  stress  should  not  be 
suppressed,  since  these  adaptations  frequently  focus  the  human  effort  on  the  selection  of 
simpler  and  more  robust  decision  strategies.  Instead,  training  should  focus  on  preparing 
decision  makers  to  handle  the  pressure  imposed  by  external  factors,  such  as  time 
constraints. 
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C.  CHAPTER  SUMMARY 

This  chapter  presented  the  main  background  information  to  help  understand  the 
cognitive  processes  and  mental  models  developed  by  AOs  in  PD  operations.  Some 
technical  concepts  were  detailed  to  stress  the  mental  calculations  used  in  periscope 
techniques,  and  give  a  dimension  of  the  cognitive  load  to  which  trainees  are  submitted 
during  periscope  drills.  Next,  the  cognitive  processes  of  PD  operations  were  analyzed 
under  the  light  of  existing  cognitive  theories,  explaining  how  it  is  suitable  to  NDM 
theories,  the  SA  building  process,  and  the  influence  of  stress  on  cognition,  highlighting 
the  use  of  PTT  as  an  approach  to  facilitate  the  execution  of  complex  tasks  in  stressful 
environments. 
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III.  LITERATURE  REVIEW 


A.  INTRODUCTION 

The  literature  review  performed  in  this  ehapter  is  divided  into  two  parts:  (1)  an 
examination  of  established  literature  about  the  general  aspeets  of  simulations  for  training, 
giving  emphasis  to  its  applieation  specifieally  for  SO  training,  and  (2)  review  of  the 
characteristies,  benefits,  and  eaveats  of  the  WBLE.  The  aim  of  this  literature  review  is  to 
answer  the  first  researeh  question:  what  are  the  advantages  of  using  a  web-based  3D 
simulation? 

B.  THE  USE  OE  SIMULATIONS  FOR  SUBMARINE  OFFICER 
TRAINING 

The  definition  of  a  simulation,  aeeording  to  the  United  States  Department  of 
Defense  is:  “a  method  for  implementing  a  model  over  time”  (Department  of  Defense, 
1994).  Inside  this  definition,  simulations  ean  assume  diverse  formats  and  be  used  for 
several  different  objectives.  Respecting  the  boundaries  of  the  scope  of  this  thesis,  the 
literature  review  will  be  focused  on  computer-based  3D  simulations  used  for  training, 
especially  when  applying  self-paced  learning  approaches,  discussing  their  general 
characteristics,  advantages,  and  disadvantages. 

1.  Simulations  for  Training 

Among  the  computer-based  simulations  used  for  training,  we  are  particularly 
interested  in  those  that  use  virtual  reality  technology.  Virtual  reality,  often  called  ‘virtual 
environments,’  is  an  interface  paradigm  using  computers  and  human-computer  interfaces 
to  create  the  effect  of  an  interactive  3D  world  (Bryson,  1996).  Brooks  (1994)  cites  four 
components  of  virtual  reality: 

•  Real  immersion  -  world  is  life  size. 

•  Real  time  -  viewpoint  changes  as  eyes  move. 

•  Real  space  -  concrete  or  abstract  3D  worlds 

•  Real  interaction  -  it  is  possible  to  manipulate  virtual  objects 
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Virtual  reality  based  simulations  (VRBS)  have  been  successfully  used  for  training 
in  several  domains,  such  as  aircraft  pilot  SA,  space  telepresence  for  astronauts,  and  the 
training  of  future  submariners  (Stone,  2002).  Facilities  with  computer-based  simulations 
working  with  full-scale  replicas  of  equipment  have  been  used  for  all  sorts  of  complex 
skills  training,  from  vehicle  operation  to  infantry  training.  These  simulations  are 
especially  useful  for  military  applications  (Salas  &  Cannon-Bowers,  2001). 

As  training  tools,  simulations  are  able  to  offer  advantages  when  compared  to 
traditional  classroom  teaching,  since  they  can  be  built  to  put  the  trainee  in  situations  and 
conditions  very  similar  to  those  one  would  face  in  the  real  world  (Chung,  2003). 
Evidence  has  been  found  that  simulations  are  effective  for  teaching  complex  skills,  and 
that,  in  training  with  simulations,  there  is  effective  transfer  of  skills  (Salas  &  Cannon- 
Bowers,  2001),  as  in  the  work  cited  from  Gopher,  Well,  and  Bareket  (1994),  in  which  the 
skills  were  transferred  from  a  computer  game-based  simulation  to  real  flight.  One  of  the 
reasons  why  simulations  are  found  more  valuable  in  contrast  with  other  training  delivery 
methods  is  that  in  simulations  the  trainee  can  experience  the  actual  performance  of  a  job 
and  its  immediate  response  (Kindley,  2002)  while  not  being  exposed  to  the  real  risks  or 
costs  of  the  real  situation  (Becker  &  Parker,  2009). 

Some  of  the  benefits  of  training  simulations,  as  cited  in  literature,  that  can  be 
directly  applicable  to  the  training  of  periscope  tasks  are: 

•  Cost  savings'.  The  costs  to  deploy  a  submarine  involve  high  numbers, 
considering  the  maintenance  of  equipment,  personnel,  and  the  deployment 
of  surface  and  air  units  to  support  exercises  (Orlansky,  Dahlman, 
Hammon,  Metzko,  Taylor,  &  Youngblut,  1994;  Thompson,  Carroll,  & 
Deaton,  2009). 

•  Replication  of  a  large  collection  of  standardized  scenarios  in  a  controlled 
situation:  Trainees  can  be  presented  with  a  large  range  of  different 
scenarios  of  the  real  world  beyond  those  presented  in  the  curriculum.  This 
is  especially  interesting  to  present  the  trainees  with  situations  that  would 
represent  high  risk  for  human  lives  or  equipment  if  performed  in  real 
situations  (Thompson  et  ah,  2009). 

•  Time  savings:  Simulations  can  compress  the  time  of  lengthy  operations, 
helping  the  trainee  to  make  the  link  between  the  action  and  consequence. 
(Chung,  2003).  For  example,  the  transit  of  the  submarine  from  one  area  of 


28 


the  exercise  to  another  can  take  a  long  and  unfruitful  time  that  can  be 
compressed  with  the  use  of  simulations  (Chung,  2003). 

•  Creation  of  a  blame-free  environment  in  which  to  make  mistakes: 
Mistakes  made  by  the  learner,  when  in  a  safe  and  blame-free  environment, 
can  be  a  powerful  learning  mechanism  (Hills,  2003).  In  exercises 
performed  at  sea  with  the  real  equipment,  the  trainees  tend  to  be  more 
limited  in  their  actions  due  to  their  fear  of  real  mistakes  and  their 
consequences. 

However,  despite  of  all  those  advantages,  it  is  important  to  keep  in  sight  that 
simulations  are  nothing  but  a  tool  (Salas  &  Burke,  2002).  The  simple  use  of  a  simulation 
for  training  is  not  a  guarantee  of  improved  learning  (Salas  &  Cannon-Bowers,  2001) 
(Salas  &  Cannon-Bowers,  2001).  Moreover,  the  advances  in  fidelity  and  resolution  do 
not  necessarily  carry  the  same  improvement  in  training  quality.  In  short,  the  simulations 
share  the  same  design  issues  and  risks  of  negative  training  as  any  other  training  tool 
(Salas,  Bowers,  &  Rhodenizer,  1998). 

2.  Simulations  Applied  to  Submarine  Officer  Training 

These  characteristics  of  simulations  intuitively  suggest  the  use  of  simulations  for 
submarine  training.  Simulations  have  already  been  a  part  of  submarine  training  for  a  long 
time;  they  have  been  combined  with  years  of  learning  in  the  classroom  and  experiences  at 
sea  (Kirschenbaum,  1989).  The  application  of  VRBS  in  submarine  training  facilitates  the 
building  of  complex  mental  models  necessary  for  the  effective  use  of  periscope 
techniques  before  the  trainees  encounter  large  simulators  with  the  mock-up 
instrumentation.  In  spite  of  possible  technology  limitations,  such  as  visual  fidelity  and 
auditory  realism,  the  benefits  added  by  the  use  of  simulations  outweigh  the  negative 
aspects  encountered  in  those  systems  (Vincenzi,  Hays,  &  Seamon,  2003). 

Indeed,  computer-based  simulations  have  been  widely  used  for  several  submarine 
trainings,  both  at  the  individual  and  team  levels  (Jones,  2008).  For  the  objectives  of  this 
thesis,  particularly  interesting  are  the  applications  of  simulations  designed  for  training 
using  personal  computers.  Those  simulations  are  generally  cost  effective  and  focused  on 
PTT,  rather  than  the  training  of  a  full  and  complex  mission  involving  the  convergence  of 
various  KSA.  Some  examples  of  these  smaller  simulations  found  in  literature  include: 
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•  Submarine  Ship  Control  Training  Program:  A  cost-effective  simulation 
implemented  on  personal  computers  for  self-paced  learning  of  the 
principles  of  submarine  ship  handling  (Biegel,  Brown,  Mason,  &  Poland, 
1998). 

•  Bottom  Gun:  A  game-based  application  developed  to  train  visual 
estimations  of  the  AOB  and  the  distance  of  surface  contacts  using  a 
periscope  simulation  (Garris  &  Ahlers,  2001). 

•  Web-based  Maritime  Training  Environment:  The  paper  describes  the 
development  of  a  web-based  simulation  (WBS)  modeled  and  developed  to 
get  around  the  high  costs  and  low  availability  of  full-scale  maritime 
simulators  (Cui,  Yicheng,  Xiuwen,  &  Yong,  2004). 

•  Submarine  Onboard  Training  (SOBT):  Among  a  list  of  simulations  used 
for  submarine  crew’s  training  cited,  the  work  elaborates  how  a  course 
originally  designed  for  four  weeks  of  classroom  and  laboratory  training, 
was  reduced  to  a  one-week  laboratory  course.  In  that  example  a  classroom 
portion  of  the  training  was  replaced  by  a  self-paced  training  onboard.  The 
training  was  a  part  of  the  SOBT  program  (Jones,  2008). 

•  Adaptive  Training  System  for  Submarine  Periscope  Operations:  The 
researchers  evaluated  the  effectiveness  of  a  periscope  simulation  in 
association  with  an  adaptive  training  system  used  for  training  of  basic 
periscope  skills  (Landsberg,  Mercado,  Van  Buskirk,  Lineberry,  & 
Steinhauser,  2012). 

A  remarkable  feature  of  those  simulations  was  the  application  of  self-paced 
learning.  In  a  field  experiment  with  novice  computer  users,  Simon  and  Werner  (1996) 
found  that  self-paced  learning  rendered  higher  cognitive  learning  and  skill  demonstration 
than  the  lecturing  approach. 

C.  USES  OF  WEB-BASED  SIMULATION  AND  WEB-BASED 
LEARNING  ENVIRONMENTS 

This  section  presents  information  from  the  literature  about  the  use  of  WBS  for 
training.  The  evolution  of  WBS  is  briefly  discussed,  citing  the  evolution  of  the  tools  that 
currently  enable  its  implementation  on  web  browsers.  Advantages  and  disadvantages  of 
WBLE  are  discussed,  stressing  the  main  human  factors  to  be  considered  in  WBLE.  Since 
the  aspects  related  specifically  to  simulations  were  discussed  in  the  previous  section,  this 
section  will  be  focused  on  the  aspects  of  WBLE.  Many  considerations  are  not  specific  to 
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WBS,  even  though,  since  WBS  is  also  a  representative  of  the  field  of  WBLE,  the 
objective  was  to  highlight  the  characteristics  of  WBLE  that  are  relevant  for  WBS. 

1.  Definitions 

The  idea  of  WBLE  involves  the  creation  of  a  meaningful  learning  environment 
with  an  instructional  program  that  takes  advantage  of  the  attributes  and  resources  of  the 
Internet  (Ritchie  &  Hoffman,  1997). 

A  WBS  is  any  representation  of  the  field  of  simulation  delivered  to  the  user 
through  the  Internet  (Huang  &  Madey,  2005).  Thus,  by  this  definition,  WBS  inherits  a 
large  number  of  applications  for  the  field  of  simulations.  There  are  WBS  for  analysis, 
testing,  etc.,  but  the  type  of  WBS  interesting  for  this  thesis  are  only  those  used  for 
training.  As  a  training  tool  that  is  delivered  to  the  users  through  the  World  Wide  Web, 
WBS  can  also  be  defined  as  WBLE,  bearing  at  the  same  time  the  features  of  VRBS  and 
WBLE.  The  hatched  area  in  the  Venn  diagram  of  Figure  6  illustrates  this  superposition. 
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Figure  6.  Venn  diagram  contextualizing  WBS  for  training. 
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2.  The  Evolution  of  Web-Based  technology 

Internet  technologies  have  been  in  constant  evolution,  since  the  early  1990s.  The 
efforts  of  software  developers  are  directed  towards  adapting  all  kinds  of  traditional 
applications  to  make  them  available  on  the  Internet  (Fishwick,  1996).  Simulations  are  not 
an  exception.  The  subject  of  WBS  was  addressed  at  a  session  in  the  Winter  Simulation 
Conference  (WSC)  in  1996  (Buss  &  Stork,  1996;  Fishwick,  1996;  Nair,  Miller,  &  Zhang, 
1996),  and  the  first  dedicated  forum  about  the  issues  related  to  WBS  was  held  at  the 
WSC  of  1998  in  San  Diego,  California.  In  the  year  of  2000,  WBS  became  a  specific  area 
of  the  WSC. 

Early  attempts  at  implementing  simulations  on  the  Internet  often  relied  on  Java 
applets  (Page,  Buss,  Fishwick,  Healy,  Nance,  &  Paul,  2000;  Harasim,  Calvert,  & 
Groeneboer,  1997;  Kuljis  &  Paul,  2000)  and  VRML  (Virtual  Reality  Modeling 
Language).  In  2011,  the  WebGL  standard  was  released,  allowing  the  rendering  of  3D 
objects  on  the  HTML5  canvas  element  of  a  browser  to  enable  accelerated  processing  of 
images  in  the  graphics  processing  unit  (GPU).  This  technology  was  not  developed  only  to 
support  the  simulations  or  virtual  environments  on  the  Internet,  but  in  this  thesis,  the 
adequacy  of  WebGL  for  this  kind  of  application  will  be  assessed. 

3.  WBLE  Advantages 

The  advantages  offered  by  any  WBLE  will  depend  on  the  intended  settings  and 
on  the  nature  of  the  training  system  designed  (Cook,  2007).  Some  advantages  that  are 
specific  to  WBLE  due  to  its  features  are: 

1.  Accessibility.  Virtually  any  place  in  which  there  is  a  device  connected  to 
the  Internet  can  become  a  potential  classroom  (Brodlie,  El-khalili,  &  Li, 
2000). 

2.  Flexible  schedule:  Not  only  are  geographical  barriers  broken,  but  also  time 
barriers.  The  WBLE  can  be  designed  to  be  available  24  hours  a  day, 
making  it  possible  for  the  learners  to  select  the  best  time  (Cook,  2007). 

3.  Low  cost:  The  end  user  generally  owns  the  training  workstation. 
Furthermore,  costs  can  be  reduced  using  available  open-source  software 
standards,  such  as  WebGL  (Brodlie  et  ah,  2000). 
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4.  Distributed  servers:  Powerful  computation  can  be  performed  by  a  remote 
server  if  needed  (Brodlie  et  al.,  2000). 

5.  Bigger  class  size:  The  application  can  be  used  at  the  same  time  by  a  larger 
set  of  students  than  would  be  possible  at  a  physical  simulator  or 
classroom.  Theoretically,  the  class  size  can  be  limited  only  by  server 
capacity  and  bandwidth  (Cook,  2007). 

6.  Generality:  A  family  of  applications  with  consistent  methodology  can  be 
made  available  for  a  range  of  different  training  procedures  (Brodlie  et  al., 
2000). 

7.  Multimedia  capabilities:  Web  technology  provides  multimedia  support 
that  can  be  used  to  add  material  to  aid  theoretical  knowledge  acquisition. 
Besides  the  simulation  itself,  the  web-technology  facilitates  inclusion  of 
tutorials.  Also,  it  is  easier  to  present  performance  results  graphically 
(Cook,  2007). 

8.  Communication  and  collaboration:  Means  of  communication  between 
students  and/or  between  students  and  instructors  can  be  made  available  to 
give  support  to  the  trainees  (Byrne,  Heavey,  &  Byrne,  2010). 

9.  Individualized  learning:  Materials  delivered  and  feedback  can  be 
individualized  for  each  trainee  by  the  application  (Cook,  2007). 

10.  Easy  updates:  Any  correction  or  update  shall  be  done  only  on  the  server, 
and  all  users  will  automatically  receive  the  new  version  (Cook,  2007). 

However,  as  it  was  mentioned  in  the  case  of  simulations,  even  though  the  web- 
based  technology  enables  several  great  opportunities  concerning  learning  environments, 
the  technology  alone  does  not  guarantee  the  learning  outcome;  good  instructional  design 
and  planning  remain  a  major  priority  (Piccoli,  Ahmad,  &  Ives,  2001). 

4.  WBLE  Disadvantages 

There  is  a  set  of  disadvantages  related  with  WBLE  as  compared  to  other  learning 
vehicles.  The  disadvantages  of  WBLE  will  be  divided  into  psychological,  technical,  and 
those  related  to  the  human-computer  interaction. 
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a.  Psychological  Drawbacks  of  WBLE 

•  Feelings  of  isolation:  All  the  flexibility  in  time  and  place  delivered  to 
trainees  also  means  that  they  will  be  frequently  studying  alone.  This 
situation,  when  kept  for  a  long  enough  time,  may  lead  some  learners  to 
feel  socially  isolated;  the  concept  of  “long”  will  depend  on  the 
characteristics  of  each  individual.  Even  with  the  use  of  online  discussion 
groups  this  feeling  may  persist  for  those  forums  that  present  a  different 
social  organization  compared  to  face-to-face  interactions  in  a  small  group, 
as  in  a  classroom.  (Berry,  2006;  Chou  &  Liu,  2005;  Cook,  2007;  Fischer, 
Spiker,  &  Riedel,  2009;  Piccoli  et  al.,  2001).  For  the  training  of  an  SO, 
AOs  play  a  key  role,  but  a  large  part  of  their  decision-making  process 
relies  on  information  processed  by  the  members  of  the  STP.  Although 
such  information  can  be  artificially  generated  by  the  simulation,  trainees 
may  feel  a  lack  of  human  interaction. 

•  Frustration:  Positive  effects  of  learner  control  can  be  neutralized  when  a 
student  experiences  frustration  derived  from  his  or  her  inability  to 
efficiently  operate  the  application  or  make  effective  instructional  decisions 
(Berry,  2006;  Chou  &  Liu,  2005;  Kara  &  Kling,  2000;  Piccoli  et  ah, 
2001). 

•  Anxiety:  When  the  WBLE  is  introduced  as  a  new  learning  method  for 
students  who  are  already  used  to  an  old  approach,  it  will  require  them  to 
navigate  through  a  new  learning  environment.  For  those  students  not  used 
to  web  technologies,  or  who  have  a  negative  attitude  toward  this  approach, 
this  kind  of  situation  may  lead  to  anxiety.  The  more  a  user  is  comfortable 
with  technology,  the  more  the  anxiety  will  give  way  to  excitement  (Chou 
&  Liu,  2005;  Fischer  et  ah,  2009;  Piccoli  et  al.,  2001;  Sitzmann,  Kraiger, 
Stewart,  &  Wisher,  2006). 

•  Confusion:  In  the  beginning  learners  used  to  old  forms  of  interaction  may 
become  confused  about  the  adaptations  (Gellman,  2005;  Kara  &  Kling, 
2000;  Piccoli  et  al.,  2001). 

•  Reduced  interest  in  subject  matter:  Students  used  to  learning  “under 
pressure”  may  lose  interest  or  feel  a  lack  of  motivation  in  a  self-paced 
environment  (Maki,  Maki,  Patterson,  &  Whittaker,  2000). 

•  Individual  differences:  Some  students  may  overestimate  their  own  ability 
and  view  less  material  than  desired  or  skip  important  components  of  the 
lessons  in  which  they  are  less  interested  (Lepper,  1985;  Reeves,  1993). 
Individuals  also  vary  the  time  required  to  find  their  best  learning  ways  to 
take  full  advantage  of  learning  control  (Milheim  &  Martin,  1991). 
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•  De-individualized  instruction:  Even  though  one  of  the  main  promises  of 
WBLE  is  the  enabling  of  a  more  individualized  instruetion,  the  opposite 
can  be  often  found  in  WBLE  that  fails  to  meet  the  user’s  individual  needs. 
It  is  easy  for  a  good  teachers  to  adapt  and  accommodate  to  different 
students’  needs,  but  the  web  application  must  be  explicitly  programmed  to 
do  so,  which  can  be  a  hard  goal  to  achieve  (Cook,  2007). 

b.  Technical  Caveats 

Other  possible  drawbacks  to  WBLE  related  to  the  technical  aspect  of  the  Internet 


are: 

•  Technical  problems:  Frequent  technical  problems,  such  as  network  issues, 
can  amplify  existing  psychological  problems  of  the  users,  increasing  their 
frustration  with  the  WBLE  (Cook,  2007). 

•  Poor  instructional  design:  A  poor  instructional  design  can  be  more  easily 
noticed  in  a  WBLE  due  to  the  absence  of  an  instructor  to  clarify  confusing 
points  and  procedures  (Cook,  2007). 

•  Technology  for  technology’s  sake:  Developers  impressed  by  the 
possibilities  of  a  new  technology  may  feel  tempted  to  implement  it  in  a 
WBLE  without  considering  if  it  is  really  useful  for  the  learning  context 
(Cook,  2007). 

•  Security  vulnerability:  Making  content  available  on  the  Internet  always 
involves  some  risks  (Byrne  et  ah,  2010). 

•  Connection  speed  issues:  Depending  on  the  application,  the  connection 
speed  of  the  user  can  influence  negatively  the  user  experience  and  the 
training  (Byrne  et  ah,  2010).  This  is  of  particular  concern  for  interoperable 
systems,  where  one  station  depends  on  the  input  of  another,  but  they  may 
be  subjected  to  different  speeds  and  latency. 

c.  Human-Computer  Interaction  Limitations 

One  of  the  advantages  of  large  training  simulators  with  physical  mock-up 
instrumentation  is  that  one  can  train  using  input  and  output  devices  similar  to  those  found 
in  the  real  system.  In  the  Submarine  Multi-Mission  Team  Trainer  (SMMTT),  for 
example,  the  simulator  is  a  room  with  the  same  layout,  and  in  many  cases  the  very  same 
equipment  one  would  find  aboard  the  vessel  (Jones,  2008).  In  that  case,  adapting  a 
simulation  to  run  over  the  Internet  using  only  a  standard  user  input  devices  would 
represent  the  loss  of  most  of  the  mechanical  interaction.  Although  these  interactions  may 
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be  important  for  the  training  of  select  skills,  some  training  segments  can  be  emulated  to 
work  with  the  forms  of  interaction  available  for  the  user  at  home.  Brodlie  et  al.  (2000) 
cites  how  haptic  devices  can  be  replaced  by  the  mouse  on  WBS  just  by  reducing  the 
degrees  of  freedom  on  object  manipulations.  To  simulate  the  force  feedback,  they  used  a 
color  code  for  the  user. 

The  most  common  forms  of  interaction  available  for  users  at  home  are: 

•  Input  devices: 

•  Mouse 

•  Keyboard 

•  Webcam 

•  Microphone 

•  Geographical  Location 

•  Touchscreen  (available  on  most  mobile  devices  as  of  2014) 

•  Gyroscopes  and  inertial  sensors  (available  on  most  mobile  devices 
as  of  2014) 

•  Output  devices: 

•  Computer  or  laptop  screen 

•  Mobile  device  screen 

•  Loudspeakers 

•  Vibration  feedback  (available  on  most  mobile  devices  as  of  2014) 

In  WBS  design,  using  creativity  and  the  results  of  scientific  studies,  developers 

can  simulate  the  human-machine  interactions  present  in  real  systems,  adapting  those 
interactions  to  be  performed  with  the  devices  available  for  the  user.  Those  adaptations 
would  allow  the  simulation  to  put  users  in  contact  with  most  of  the  interactions  they  will 
later  experience  in  the  actual  system.  However,  it  is  important  to  avoid  the  introduction  of 
the  negative  training,  using  a  form  of  interaction  too  different  from  that  of  the  real 
environment.  In  some  cases,  it  might  be  better  to  design  the  WBLE  to  perform  only  PTT. 
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d. 


Health  Issues 


As  is  common  for  virtual  environments,  users  may  experienee  some  cyber¬ 
sickness  (Stanney,  Kennedy,  &  Drexler,  1997).  This  eondition  ean  be  aggravated  by  the 
faet  that  the  virtual  environment  movements  in  a  WBS  do  not  aecompany  actual 
movement  of  the  user’s  body.  Designers  of  WBS  should  employ  different  techniques  to 
reduee  those  undesired  effeets  and  make  the  experience  more  eomfortable  for  the  learner. 
Other  possible  health  issues  are  the  ones  eommon  to  normal  use  of  the  Internet,  such  as 
addiction,  depression,  etc.,  which  are  out  of  the  scope  of  this  thesis. 

5.  Pedagogical  Factors  of  WBLE 

Most  authors  assume  that  WBLE  learning  philosophy  and  the  interaetive  nature  of 
that  software  are  aligned  with  the  eonstruetivist  theory  (Piccoli  et  ah,  2001).  The 
constructivist  learning  theory  asserts  that  individuals  take  more  advantage  of  learning 
when  they  diseover  things  by  themselves,  while  eontrolling  the  pace  of  learning  and 
using  the  support  of  instruetors  only  when  it  is  needed.  This  view  is  in  opposition  to  the 
objectivist  approach  that  assumes  the  pre-existence  of  the  knowledge  presented  to  the 
student  by  the  instructor,  and  that  the  instruetor  is  the  owner  or  eontroller  of  that 
knowledge  (Ahmad,  Piceoli,  &  Ives,  1998). 

When  the  effeets  of  WBLE  are  being  considered,  two  important  pedagogieal 
concepts  frequently  eome  into  play:  learner  self-efficaey  and  learner  eontrol.  Bandura 
(1986)  defines  self-efficacy  as: 

. . .  people’s  judgments  of  their  capabilities  to  organize  and  execute  eourses 
of  aetion  required  to  attain  designated  types  of  performanees.  It  is 
concerned  not  with  skills  one  has  but  with  judgments  of  what  one  can  do 
with  whatever  one  possesses,  (p.  391). 

The  main  coneem  of  self-efficaey  is  the  judgment  of  what  the  person  ean  do  with 

what  he  or  she  possesses.  In  the  context  of  WBLE  the  importance  of  self-efficacy  may 

indicate  the  propensity  of  the  learners  to  aetually  apply  what  they  have  learned.  Higher 

levels  of  self-effieaey  have  important  influenee  on  individuals’  behavior  (Bandura,  1986). 

According  to  Compeau  and  Higgins  (1995),  self-efficacy  can  influence  the  learner’s 

choice  about  the  behavior  to  undertake,  effort  to  exert,  and  persistenee  in  attempting 
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those  behaviors.  Self-efficacy  also  helps  learners  to  adjust  and  handle  an  unfamiliar 
learning  environment. 

Learner  control  is  the  degree  of  freedom  that  students  can  exert  over  the  pace, 
sequence,  and  content  of  instruction  in  a  learning  environment  (Milheim  &  Martin, 
1991).  Learner  control  is  important  because  it  is  believed  to  increase  the  learner’s 
satisfaction,  consequently  also  increasing  the  learner’s  self-efficacy.  Learner  control  has  a 
potential  to  turn  into  satisfaction,  enjoyment,  and  confidence,  and  it  makes  students  feel 
more  efficacious  (Taipjutorus,  Hansen,  &  Brown,  2012).  Technology  mediated  learning 
environments  improve  both  the  student’s  evaluation  of  the  learning  experience  and  his  or 
her  achievements.  It  also  helps  to  increase  teacher-student  interaction  and  make  the 
learning  process  more  student-centered  (Piccoli  et  al.,  2001). 

6.  Adaptive  and  Intelligent  Tutoring  Systems 

Web-based  simulations  convey  to  the  learner  a  high  flexibility,  which  mainly 
derives  from  the  fact  that  they  can  be  accessed  anywhere,  anytime,  depending  only  on  the 
user’s  schedule.  From  this  same  feature,  however,  derives  one  of  the  biggest  caveats  of 
training  systems  over  the  Internet:  a  lack  of  instructors’  attention  to  monitor  students’ 
progress.  Although  simulations  are  undoubtedly  useful  as  training  tools,  the  simulations 
themselves  are  not  sufficient  to  guarantee  a  valuable  and  thorough  learning  experience.  If 
only  the  scenario  is  presented,  without  doctrinal  and  procedural  guidance,  even  an 
experienced  user  can  become  lost.  Simulations  designed  for  training  purposes  are  more 
effective  when  they  come  alongside  with  instructional  features  embedded  in  them  (Salas 
&  Burke,  2002).  Also,  the  effectiveness  of  a  training  simulation  is  improved  when  it 
provides  an  assessment  of  performance,  not  only  to  ease  the  evaluation  of  the  instructor 
and  avoid  subjectivity,  but  also  to  provide  diagnosis  on  skills  deficiencies  (Salas  & 
Burke,  2002).  A  powerful  approach  to  training  is  one-on-one  tutoring  (Bloom,  1984),  in 
which  one  instructor  follows  one  student  closely,  correcting  weaknesses  and  reinforcing 
strong  points.  Most  of  the  strength  of  this  approach,  in  comparison  to  group  training  or 
self-tutoring,  comes  from  the  fact  that  human  instructors  are  able  to  scaffold  the  student 
during  the  learning  process  and  give  feedback  to  encourage  the  student’s  progress 
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(VanLehn,  201 1).  Nonetheless,  this  teaching  style  is  generally  not  feasible  in  the  field  of 
military  training,  where  normally  there  are  only  a  few  instructors  to  teach  a  relatively  big 
group  of  trainees  within  a  limited  time  frame.  On  the  other  hand,  since  a  web-based 
application  is  frequently  a  lonely  adventure,  it  carries  all  the  caveats  of  any  self-education 
experience.  Unguided  practice  can  often  lead  to  negative  training,  loss  of  focus  on  the 
right  behaviors,  bad  distribution  of  time  among  the  various  aspects  of  training,  and 
diminishing  of  training  transfer  to  the  job  (Salas  &  Burke,  2002).  Those  characteristics 
give  a  clue  that  perhaps  the  use  of  intelligent  tutoring  systems  (ITS)  can  be  a  leveraging 
tool  for  WBS.  Some  aspects  of  the  one-on-one  tutoring  approach  can  be  reproduced  and 
improved  in  computer-based  training  (CBT)  using  ITS.  ITS  can  help  to  fix  some  of  the 
main  drawbacks  of  CBT  and  self-tutoring  (Landsberg  et  al.,  2012).  ITS  is  meant  to 
provide  immediate  and  customized  instruction  or  feedback  to  learners  (Psotka,  Massey, 
&  Mutter,  1988),  while  delivering  adaptive  learning  in  which  the  training  interventions 
can  be  tailored  to  individual  constraints,  aptitudes,  and  preferences.  The  review 
conducted  by  Van  Lehn  (2011)  in  several  experiments  about  the  effectiveness  of  ITS,  has 
found  that  ITS  is  nearly  as  effective  as  one-on-one  tutoring,  with  an  effect  size  of  d  = 
0.76  for  ITS  in  opposition  to  d  =  0.79  for  the  one-on-one  approach.  It  is  indispensable  to 
emphasize,  though,  that  the  studies  reviewed  covered  several  different  types  of  skills  and 
audiences.  The  correlation  between  specific  types  of  skills  and  the  efficiency  of  ITS 
training  was  not  the  focus  of  Van  Lehn’s  study. 

7.  WBLE  Effectiveness  Findings 

There  are  several  studies  that  compared  WBLE  and  e-leaming  with  the  traditional 
classroom  environment.  Chou  and  Liu  (2005)  in  their  research  with  junior  high  school 
students  found  that  students  using  a  WBLE  reported  higher  learning  performance,  higher 
computer  self-efficacy,  higher  satisfaction,  and  higher  levels  of  learning  climate.  Their 
field  experiment  focused  on  the  effectiveness  of  WBLE  for  basic  information  technology 
skills  training.  In  another  research  conducted  by  Sitzmann,  Kraiger,  Stewart,  and  Wisher 
(2006),  traditional  classroom  and  WBLE  were  found  to  have  the  same  effectiveness  for 
teaching  procedural  knowledge  and  for  teaching  declarative  knowledge  when  the  same 

instructional  method  was  used  for  both,  highlighting  the  overriding  importance  of 
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instructional  design.  However,  in  the  same  study,  the  instruction  through  WBLE  was 
found  to  be  19  percent  more  effective  than  classroom  instruction  for  teaching  declarative 
knowledge  when  trainees  received  feedback  in  long  courses  and  when  learner  control  was 
provided. 

WBLE  includes  an  overarching  range  of  instructional  styles  and  cannot  be  treated 
as  a  single  entity  (Cook,  Garside,  Levinson,  Dupras,  &  Montori,  2010).  WBS  is  only  one 
of  the  instructional  styles  of  WBLE.  WBS  can  potentially  merge  the  advantages  of 
WBLE  and  simulations.  In  a  meta-analysis  of  the  instructional  effectiveness,  simulation 
games  were  found  to  deliver  14  percent  more  effectiveness  for  procedural  knowledge 
teaching  and  1 1  percent  more  effectiveness  for  declarative  knowledge,  with  9  percent 
more  retention,  and  20  percent  higher  self-efficacy,  showing  that  this  modality  of 
teaching  has  the  potential  to  leverage  the  teaching  of  work-related  knowledge  and  skills 
(Sitzmann,  2011). 

Although  several  studies  have  found  evidence  about  the  effectiveness  of  WBLE 
for  teaching,  a  question  always  to  be  considered  is  whether  the  increased  effectiveness  is 
accounted  for  by  the  characteristics  of  WBLE  or  by  the  suitability  of  the  instructional 
method  when  applied  via  WBLE.  Clark  (1994)  states  that,  although  some  media  may  be 
more  cost-effective,  the  final  effectiveness  of  learning  is  not  influenced  at  all  by  the 
media,  which  works  only  as  a  vehicle.  Reeves  and  Reeves  (1997)  complement  this  view, 
listing  ten  dimensions  that  will  actually  influence  the  effectiveness  of  interactive  learning 
in  WBLE: 

1 .  Pedagogical  Philosophy.  The  adoption  and  application  of  an  instructivist 
or  constructivist  epistemological  approach. 

2.  Learning  Theory.  The  predominance  in  instructional  design  of  either  a 
behavioral  or  cognitive  theory.  Behavioral  theory  focuses  instruction  on 
shaping  the  subject’s  behavior  to  identified  desirable  behaviors  through 
stimuli,  responses,  feedback,  and  reinforcement.  The  cognitive  theory  puts 
more  emphasis  on  internal  mental  states  (mental  models,  skills,  schema, 
rules)  than  on  externally  manifested  behavior. 

3.  Goal  Orientation:  The  instructional  goals  can  range  from  sharply  focused 
to  general  goals. 
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4.  Task  Orientation:  Whether  the  learning  is  more  geared  to  aeademie  or 
authentie  (practical)  tasks. 

5.  Source  of  Motivation:  Can  be  predominantly  extrinsic,  meaning  that  the 
motivation  to  learn  comes  from  outside  the  learning  environment,  or 
intrinsic,  when  the  motivation  is  built  by  the  learning  environment. 

6.  Teacher  Role:  Ranging  from  a  didactic  (direct  participation)  to  facilitative. 

7.  Metacognitive  Support:  Whether  the  learning  system  is  supportive  or  not; 
“the  skills  one  has  in  learning  to  learn”  (p.  62). 

8.  Collaborative  Learning:  WBLE  can  be  designed  to  ignore  or  promote 
collaborative  learning. 

9.  Cultural  Sensitivity:  The  system  may  be  insensitive  to  or  respectful  of 
cultural  implications  of  the  tasks  and  subject  matters  covered. 

10.  Structural  Flexibility:  Whether  the  system  is  fixed  or  open  regarding  time 
and/or  place. 

The  adjustment  of  each  dimension  of  this  model  for  each  training  need  would  lead 
to  the  optimization  of  the  learning  effectiveness  (Reeves  &  Reeves,  1997). 

D.  CHAPTER  SUMMARY 

When  deciding  about  the  introduction  of  a  novel  technology  for  training  support, 
one  should  keep  in  mind  that  it  is  the  way  in  which  the  instruction  is  implemented  while 
using  that  technology  and  not  the  technology  itself  that  determines  the  improvement  or 
deterioration  of  learning  outcomes  (Clark,  1994;  Piccoli  et  ah,  2001).  Although  the 
appropriateness  of  a  WBS  depends  on  factors  like  instructional  context,  objectives,  and 
instructional  approach,  the  technology  enables  the  use  of  tools  to  aid  learning  that  would 
be  impossible  in  other  environments  or  with  other  type  of  technology.  Flexible  WBS  can 
be  adapted  to  supplement  thorough  simulations.  This  approach  can  be  used  to  address 
isolated  skills  and  abilities  using  the  browser  and  reduce  the  time  necessary  for  training 
ashore.  The  use  of  PTT  is  a  key  approach  for  WBS.  Creative  solutions  can  be  deployed  to 
replace  missing  interactions  available  on  the  original  simulator.  The  absence  of 
instructors  can  be  remedied  with  the  application  of  tutoring  powered  by  artificial 
intelligence  and  help  avoid  negative  learning.  Submarines  are  complex  and  very 
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expensive  weapons,  and  their  operation  requires  very  well-prepared  people.  Not  paying 
due  attention  to  the  training  of  submarine  personnel  risks  the  loss  of  lives  and  assets. 
Simulators  have  substantially  enhanced  the  quality  of  training,  but  they  have  constraints 
related  to  their  cost  and  availability.  Web-based  simulations,  though,  if  well  designed  and 
applied,  could  complement  the  training  process  that  use  simulations  and  complement  or 
replace  part  of  the  time  needed  for  classroom  instruction.  WBS  is  especially  useful  as  a 
tool  to  help  trainees  develop  the  skills  that  they  would  otherwise  need  to  practice  without 
any  help.  Regarding  the  time  of  use  of  WBS,  this  type  of  training  solution  can  be  used 
before,  during,  or  after  the  programmed  career  courses  or  training  sessions.  If  used  before 
the  courses,  they  can  prepare  the  students  with  basic  skills  and  solve  minor  deficiencies 
upfront,  so  that  students’  time  in  the  training  facility  will  be  optimized.  If  used  during  the 
career  courses,  they  can  teach  or  reinforce  necessary  knowledge.  If  used  after  the  student 
has  had  formal  training,  those  systems  keep  the  student  up-to-date  with  his  or  her  abilities 
while  offering  a  flexibility  of  time  and  place  where  training  will  be  organized.  It  is 
important  to  note  that  adapting  an  existing  simulation  to  use  the  support  of  WBLE  might 
require  ongoing  revision  to  minimize  the  effect  of  disadvantages  and  take  full  advantage 
of  the  strong  points  that  the  web-based  approach  offers. 
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IV.  TASK  ANALYSIS 


The  goal  of  the  task  analysis  was  to  provide  a  detailed  description  of  the  tasks 
done  by  an  AO  and  to  identify  the  parameters  that  would  guide  the  development  of  the 
WEBS.  This  task  analysis  also  aimed  to  provide  the  answers  to  research  questions  2  and 
3: 

Question  2.  What  are  the  main  KSA  necessary  for  the  performance  of  the  AO 
and  his  role  in  a  submarine? 

Question  3.  Which  of  those  KSA  could  be  improved  with  the  help  of  a  WBLE, 
without  the  direct  help  of  an  instructor  and  outside  the  classroom 
environment? 

The  execution  of  the  task  analysis  is  performed  in  three  steps:  description  of  the 
missions,  analysis  of  the  tasks  involved  in  the  missions,  and  selection  of  relevant 
information  from  the  task  analysis. 

In  the  first  step,  using  the  AT  from  the  BNSS  as  an  example,  the  main  training 
missions  performed  in  training  sections  at  that  simulator  are  described.  The  goal  was  to 
build  an  overall  vision  of  the  tasks  that  are  usually  trained  at  these  simulators. 

Next,  a  single  mission  is  selected  to  perform  the  task  analysis.  The  Cognitive 
Task  Analysis  (CTA)  methodology  is  used  to  decompose  the  mission  in  tasks,  identifying 
goals,  tasks,  and  sub-tasks,  MOP,  and  the  KSA  required  to  perform  those  tasks. 

The  last  step  used  information  generated  by  the  CTA  and  determined  which  set  of 
tasks  could  be  implemented  in  the  WEBS  system  in  the  form  of  PTT. 

The  main  point  considered  for  planning  the  design  of  WEBS  is  that  it  is 
impossible  to  replicate  all  the  interfaces  available  at  the  AT  in  the  users’  home.  Thus,  the 
tasks  were  detailed  and  subdivided,  having  the  KSA  identified  to  enable  their  training 
separately,  following  the  PTT  approach. 
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A.  DESCRIPTION  OF  THE  MISSIONS 

The  first  step  consisted  of  understanding  the  missions  performed  in  the  original 
simulator.  This  section  describes  the  main  missions  usually  trained  in  courses 
administered  at  the  AT  of  BNSS.  The  objective  is  to  yield  a  holistic  view  of  the  context 
in  which  the  tasks  are  performed. 

The  ATs  are  modular  simulators  built  in  a  facility  at  BNSS.  They  provide  a  mock- 
up  of  the  equipment  found  in  a  submarine  Control  Room.  There  are  four  rooms  that 
compose  each  facility.  The  first  room  houses  only  the  computer  processors  that  generate 
input  signals  for  the  simulation,  acting  the  role  of  sensors.  The  second  room  is  the 
instructor  management  room  where  the  operators  can  control  the  simulation  and  record 
the  trainee’s  performance.  There  is  also  an  attached  classroom  used  for  briefing  and 
debriefing  of  exercises.  Finally,  there  is  the  simulator  room  which  represents  a 
reconstruction  of  the  Control  Room  of  a  “Tupi”  or  “Tikuna”  class  submarine.  All  the 
stations  manned  by  the  members  of  the  STP  and  respective  equipment  replicate  the  same 
layout  found  in  the  submarine  Control  Room  (Figure  7).  The  AT  can  provide  team 
training  for  an  entire  STP  of  a  submarine,  and  it  also  has  a  permanent  crew  to  man  it 
during  courses  and  specific  trainings  when  the  training  is  focused  only  on  the  AO.  The 
role  of  the  key  elements  of  the  STP  is  summarized  in  Table  3. 


Table  3.  Main  responsibilities  of  the  key  members  in  the  Section  Tracking 

Party. 


Team  Member 

Responsibility 

Executive  Officer  (XO) 

Manages  the  team,  combines  the  solutions 
of  all  stations  and  generates  a  general  STP 
solution 

Sonar  Supervisor 

Manages  detection,  tracking, 
classification,  and  solutions  provided  by 
the  Sonar  Systems 

Combat  Systems  Supervisor 

Manages  the  combat  systems  and  the 
solutions  provided  by  them 

Contact  Evaluation  Plot  (CEP)  Supervisor 

Manages  solutions  of  Target  Motion 
Analysis  (TMA)  using  the  CEP. 

Plotting  Supervisor 

Manages  the  geographical  plotting  and  the 
TMA  using  the  plotting  table  methods. 
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Some  of  the  missions  usually  trained  at  the  AT  are: 

1 .  Go-Deep  Range  Exercise  (GODEX) 

2.  Anti-surface  warfare 

3 .  S  econdary  Operations 

•  Intelligence,  surveillance,  and  reconnaissance  (ISR) 

•  Mine  warfare 

•  Special  operations 

4.  Evolution  to  Periscope  Depth 

5.  Anti-submarine  warfare  (Sub  x  Sub) 


Figure  7.  Positioning  of  the  Section  Tracking  Party  in  the  Control  Room  and 

at  the  Attack  Trainer  Simulator. 
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1.  Go-Deep  Circle  Exercise  (GODEX) 

The  GODEX  is  used  to  train  and  assess  AOs  on  their  mastering  of  the  coneepts  of 
submarine  safety  in  PD.  This  training,  also  known  as  “eyes-only,”  is  generally  applied  as 
a  preliminary  step  to  eertify  that  the  trainee  is  able  to  employ  PD  attaek  rules  and  keep 
the  submarine  safe  in  more  complex  operations.  In  order  to  exercise  most  of  the  precepts 
of  safety,  it  exposes  the  trainees  to  extreme  and  highly  stressful  situations.  Such  extreme 
situations  involve  the  approximation  of  ASW  ships  to  the  submarine  in  varied 
configurations;  sometimes  they  are  made  to  be  passing  close  to  the  GDC  limit,  and 
sometimes  crossing  it  and  passing  over  the  submarine.  An  example  of  a  run  with  two 
contacts  is  shown  in  Figure  8. 

The  AO  is  required  to  keep  SA  all  the  time  with  the  lowest  possible  collection  of 
data  with  the  periscope  and  following  strict  procedures  to  diminish  mast’s  exposure.  If 
the  trainee  judges  that  the  safety  of  the  submarine  is  compromised  by  the  proximity  of  a 
contact,  the  only  maneuver  allowed  is  to  descend  to  SD,  wait  for  the  contact  to  pass  over 
the  submarine,  reassess  the  situation,  and  then,  if  the  situation  allows  it,  evolve  to  PD. 

The  trainee  is  not  allowed  to  change  the  course  or  speed  of  the  submarine  during 
the  exercise.  While  this  situation  is  very  unlikely  in  a  real  combat  scenario,  the  training  of 
all  the  concepts  of  safety  at  PD  does  not  happen  in  just  one  exercise;  a  series  of  training 
sessions  are  used  to  exercise  and  assess  the  trainees’  mental  quickness,  emotional  control, 
and  cognition  capability  under  stress.  Another  objective  is  to  promote  the  trainees’  self- 
confidence  and  force  them  to  make  contact  with  their  own  limitations  and  capabilities. 
This  exercise  is  performed  in  the  same  manner  both  at  the  simulator  and  at  sea  in  the  real 
environment. 
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Figure  8.  Example  of  relative  motion  plot  of  a  GODEX  run  with  two  contacts. 

The  exercise  begins  with  the  submarine  at  PD,  crossing  a  SPA  where  the  traffic  of 
surface  ships  is  expected.  The  trainee  assumes  the  AO  role,  and  is  assigned  the  task  to 
maintain  the  submarine  at  PD  as  long  as  possible,  without  jeopardizing  the  submarine  and 
allowing  the  surface  contacts  to  reach  its  GDC.  All  calculations  must  be  made  mentally 
by  the  AO  using  several  stopwatches  to  control  time  for  each  contact  and  for  the  ARE 
interval. 

2.  Anti-Surface  Warfare 

The  main  mission  for  a  conventional  submarine  is  to  attack  a  surface  ship, 
generally  the  High  Value  Unit  (HVU)  of  an  enemy  task  force  like  a  war  vessel  with  high 
strategic  relevance.  Due  to  the  importance  of  these  ships  and  their  relative 
defenselessness,  HVUs  are  generally  escorted  by  ASW  ships,  aircrafts  and  even  hunter- 
killer  submarines,  making  it  very  challenging  for  the  submarine  to  approach  and  attack. 
These  escorts  are  all  endowed  with  accurate  sensors  and  powerful  anti-submarine 
weapons.  If  the  personnel  of  the  submarine  want  to  successfully  execute  an  attack,  the 
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submarine  must  be  previously  positioned  to  approach  the  objective  target  at  the  right  time 
and  at  the  right  place,  while  at  the  same  time  does  its  best  to  avoid  detection  by  ASW 
units.  This  kind  of  mission  does  not  necessarily  involve  operations  on  PD. 

3.  Secondary  Operations 

The  missions  of  ISR,  mine  warfare,  and  special  operations  are  henceforth  named 
secondary  operations.  The  approach  phase  of  these  missions  is  quite  similar  to  that  of 
ASW.  These  missions  will  generally  differ  from  ASW  only  in  terms  of  the  final 
objective.  Since  the  main  interest  of  this  thesis  is  on  the  approach  phase;  the 
particularities  of  each  of  these  missions  will  not  be  detailed. 

4.  Evolution  to  Periscope  Depth 

The  PD  evolution  is  a  procedure  composed  of  a  series  of  actions  executed  to 
move  the  submarine  to  PD,  coming  from  a  deeper  depth.  To  evolve  to  PD,  the  trainee,  in 
the  role  of  a  watch  officer,  commands  a  series  of  procedures  to  collect  information  about 
surface  contacts  and  the  surrounding  environment,  and  then  chooses  the  best  course  to 
take  the  submarine  to  PD.  Almost  all  tasks  are  executed  with  the  submarine  below  PD, 
and  therefore  they  are  outside  of  the  scope  of  this  thesis. 

5.  Anti-submarine  Warfare 

This  mission  involves  hunting  of  another  submerged  submarine.  All  the  actions 
are  generally  taken  with  the  submarine  transiting  deep  that  is  done  without  the  use  of  the 
periscope;  as  such  it  is  outside  of  the  scope  of  this  thesis. 

B.  COGNITIVE  TASK  ANALYSIS  (CTA) 

The  GODEX  mission  was  selected  for  the  CTA,  because  it  is  the  most  intensive 
and  comprehensive  mission  regarding  the  training  of  periscope  techniques.  CTA 
encompasses  a  series  of  methods  that  study  and  describe  the  knowledge  and  reasoning 
used  while  performing  some  tasks.  It  is  useful  when  it  is  necessary  to  elicit  the 
organization  and  structure  of  information  that  people  use,  to  understand  how  they  think. 
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what  they  know,  and  what  they  seek  to  understand  better  (Crandall,  Klein,  &  Hoffman, 
2006). 

The  CTA  was  performed  to  pinpoint  the  tasks  that  could  be  simulated  in  WEBS, 
to  list  the  critical  cues  that  should  be  provided,  and  the  set  of  KS A  required  for  each  task. 
As  important  as  it  is  to  identify  the  tasks  to  be  simulated,  it  is  also  important  to  identify 
and  prune  the  tasks  that  are  not  recommended  to  train  on  a  WBS  because  they  are  too 
complex,  or  they  could  lead  to  negative  training  due  to  the  lack  of  appropriate  interface  or 
sensory  cues.  A  good  example  of  tasks  not  intended  to  be  trained  are  those  related  to  the 
operation  of  the  periscope  controls.  Although  it  is  important  for  the  trainees  to  get  used  to 
these  controls,  these  controls  are  very  specific  to  each  equipment  type,  and  should  be 
trained  on  a  replica  or  at  the  equipment  itself  The  attempt  to  virtually  recreate  it  in 
WEBS  could  lead  to  negative  training  transfer. 

Most  of  the  sources  of  documentation  for  the  techniques  used  in  these  tasks  are 
classified.  This  fact  has  limited  considerably  the  scope  of  the  CTA  reported  in  this  thesis. 
The  task  analysis  will  only  reproduce  the  level  of  detail  that  can  be  found  in  non- 
classified  sources.  As  the  usability  study  was  performed  publicly,  only  a  part  of  the  ARE 
task  was  selected  to  be  implemented  in  WEBS  and  published  for  the  usability  study. 

The  CTA  was  performed  in  four  steps,  using  the  same  methodology  successfully 
employed  by  Moraes  (2011)  for  the  design  of  a  game-based  shiphandling  simulator: 

1.  Detail  and  decompose  tasks  performing  a  Hierarchical  Task  Analysis 
(HTA) 

2.  Generate  a  Critical  Cues  Inventory  (CCI) 

3.  Select  tasks  to  be  trained 

4.  Identify  MOP 

The  steps  are  detailed  in  the  next  sections.  The  sources  used  were  the  observation 
of  videos  of  trainees  executing  the  GODEX  mission,  and  the  analysis  of  all  documents. 


49 


1.  Hierarchical  Task  Analysis  (HTA) 

The  first  step  consisted  in  performing  a  HTA.  The  HTA  is  a  way  of  representing 
hierarchically  the  goals  and  sub-goals  of  a  system,  highlighting  its  operations  and  plans, 
to  allow  an  extended  analysis  (Stanton,  2006).  Performing  a  HTA  helps  to  identify  the 
cognitive  tasks  driven  by  these  goals  (Annett,  2003).  Following  the  methodology  and 
notation  proposed  by  Stanton  (2006),  a  simplified  diagram  for  the  GODEX  mission,  with 
the  hierarchy  of  tasks  and  plans  of  execution,  is  summarized  in  Figure  9. 


Figure  9.  Hierarchical  tasks  tree  diagram  for  the  GODEX  mission. 


The  procedural  diagram  in  Figure  10  emphasizes  some  points  of  decision  in  the 
mission,  detailing  it  up  to  the  third  level  of  sub-tasks. 

A  detailed  description  of  the  tasks  from  the  HTA  diagram  was  implemented  in  the 
tabular  form.  The  detailed  description  includes  valuable  information  about  the  cognitive 
processes  and  sensory  perception  performed  by  the  AO,  also  stressing  some  key  required 
KSA.  Most  of  the  tasks  rely  on  doctrinal  rules  based  on  classified  sources. 
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1.2  -  Manoeuver 


r 


Report 

Results 


No 


Figure  10.  Summarized  procedural  diagram  for  GODEX  mission. 

Due  to  its  simplicity  and  lack  of  classified  information,  the  ARE  task  (task 
1.1. 2. 2)  was  selected  as  an  example  for  implementation  in  WEBS,  and  it  is  used  for  the 
usability  study.  Figure  11  shows  the  hierarchical  diagram  of  this  task.  The  detailed 
tabular  form  for  this  task  is  reproduced  in  Table  4  (note  that  only  the  tasks  that  do  not 
involve  any  classified  source  are  reproduced  in  this  table). 
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Figure  1 1 .  Hierarchical  diagram  of  sub-tasks’  tree  for  the  ARL  task. 
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Table  4.  Detailed  description  of  tasks. 


Step 

Task 

Task  Description 

Remarks 

1 

Traverse 

Area  in 

Safety 

This  is  the  overall  goal.  The 
AO  must  traverse  an  area  in 
which  there  will  be  surface 
ships  transiting.  For  the 
version  considered  in  this 
study  the  trainee  is  not 
allowed  to  change  the 
submarine’s  course. 

During  the  traversal  the 
trainee  must  keep  the 
periscope  lowered  for  as 
long  as  possible  without 
risking  the  safety  of  the 
submarine. 

1.1 

Update 

Tactical 

Situation 

AO  performs  the  most 
adequate  technique  to  obtain 
information  about  the 
surrounding  tactical  situation. 

The  AO  must  assess  the 
source  of  information  to 
be  evaluated  based  on  his 
mental  model  of  the 
situation  and  on  the  time 
available  according  to  his 
control  of  time. 

1.1.1 

Evaluation  of 

Sensors’ 

Information 

AO  obtains  information  of  the 
contacts  straight  from  sensors’ 
displays  or  from  supervisors. 

AO  checks  if  the 
information  from  sensors 
is  consistent  with  his 
model  of  the  situation. 

Evaluate  data 
processed  by 
STD 

AO  checks  the  STD  solutions 
managed  by  the  XO. 

AO  compares  solutions 
with  his  own  mental 
models 

1.1.2 

Data 

Collection 

with 

Periscope 

AO  obtains  information  of  the 
contacts  performing  a 
periscope  observation. 

AO  must  judge  which 
type  of  periscope 
observation  is  appropriate 
based  on  the  data  he  needs 
to  obtain  and  the  time  he 
has  to  do  it. 

1.1. 2.2 

All  Round 
Look 

AO  performs  an  ARE. 

1.1.2.2.1 

Pre- 

Observation 

Actions 

Set  of  tasks  performed  to  put 
the  periscope  ready  for  an 
observation.  Comprises  all 
actions  taken  from  the 
decision  to  raise  the  periscope 
until  the  AO  puts  the  eyes  on 
the  eyepieces  to  begin  the 
observation. 
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Step 

Task 

Task  Description 

Remarks 

1.1.2.2.1.1 

Check 

Platform 

Conditions 

Conditions  of  the  submarine 
to  be  checked  every  time 
before  the  periscope  is  raised. 

Each  condition  should  be 
checked  “at  a  glance.” 
Depending  on  conditions 
like  the  depth  of  the  keel, 
speed  over  ground,  or  hull 
inclination,  the  AO  may 
decide  to  not  raise  the 
periscope  until  corrections 
are  made  on  the  platform 
conditions. 

1.1.2.2.1.2 

Communicat 
e  intention 

After  deciding  that  all 
conditions  allow  for  periscope 
raising,  the  AO  announces 
orally  to  STD  what  kind  of 
observation  he  will  do. 

1.1.2.2.1.3 

Command 
Periscope  to 
Raise 

AO  commands  the  periscope 
he  will  use  to  be  raised 

After  the  AO  commands 
the  periscope  to  be  raised, 
the  CEP  supervisor  orally 
announces  the  estimated 
bearing  of  known 
contacts. 

1.1.2.2.1.4 

Set  Periscope 
Bearing 

AO  turns  the  periscope  to  the 
bearing  where  he  expects  to 
begin  his  observation. 

Assessing  his  SA  and  the 
information  just  received 
from  CEP  Supervisor,  the 
AO  decides  what  is  the 
best  bearing,  according  to 
safety  rules,  to  begin  the 
ARE. 

As  periscope  comes  closer 
to  the  floor  limit,  the  AO 
crouches. 

As  the  periscope  handles 
become  reachable  near  the 
floor,  the  AO  turns  it  to 
the  selected  bearing. 

1.1.2.2.1.5 

Check/Set 

Periscope 

Controls 

AO  manually  sets  the 
periscope  controls  while  it  is 
being  raised 

To  save  time,  the  AO 
must  crouch  and  do  these 
settings  as  soon  as  he  can 
reach  the  periscope 
controls  near  the  floor, 
while  he  still  cannot  reach 
the  eyepieces  with  his 
eyes. 

54 


Step 

Task 

Task  Description 

Remarks 

I.I.2.2.2.5. 

1 

Check/Set 

Zoom 

The  periscope  zoom  must  be 
set  in  accordance  to  the  type 
of  observation  to  be  made. 

AO  sets  the  periscope  to 
the  lowest  zoom  to 
perform  an  ARL. 

1.1.2.2.2 

Observation 

Execute  Observation 

AO  execute  chosen  type 
of  observation. 

1.1.2.2.2.1 

Set  Periscope 
Height 

AO  sets  the  periscope  height. 

AO  evaluates,  according 
to  his  own  height  and  sea 
state,  a  height  for  the 
periscope  in  which  it  is 
possible  to  execute  the 
observation,  exposing  just 
the  sufficient  amount  of 

mast. 

1.1.2.2.2.2 

Set  Periscope 
Lens 

Elevation 

AO  uses  periscope  controls  to 
set  the  best  lens  elevation  to 
execute  the  search. 

AO  must  recall  safety 
doctrine  to  adjust  it, 
considering  expected 
threat. 

1.1.2.2.2.3 

Make  visual 
search 

AO  turns  the  periscope  360'’, 
visually  scanning  the  horizon 
looking  for  contacts. 

AO  must  focus  total 
attention  to  this  search. 

1.1.2.2.3 

Post 

Observation 

Actions 

AO  takes  action  to  lower  the 
periscope  just  after  finishing 
the  observation  procedure. 

According  to  the  current 
tactical  situation,  the  AO 
must  predict  what  is  his 
next  step,  and  decide  to 
lower  the  periscope  all  the 
way  down  or  just  enough 
to  break  its  exposition 
(flood  the  periscope) 
allowing  for  a  faster 
raising  afterwards. 

1.1.2.2.3.1 

Command 
Periscope  to 
Lower 

AO  commands  orally  to  lower 
or  “flood”  the  periscope. 

AO  commands  orally  to 
lower  or  “flood”  the 
periscope 
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Step 

Task 

Task  Description 

Remarks 

1.1.2.2.3.2 

Reposition 

Periscope 

While  the  periscope  is  being 
lowered,  the  AO  quickly  turns 
it  to  the  start  bearing  of  the 
next  observation. 

Having  a  refreshed  SA 
after  the  execution  of  an 
ART,  the  AO  must  plan 
one  step  ahead,  choosing 
the  next  periscope  to  be 
done,  and  setting  the 
periscope  bearing  on  the 
bearing  to  begin  the  next 
observation.  This 
movement  must  be  done 
quickly  and  without 
looking  at  the  eyepieces. 

1.1.2.2.4 

Report 

Results 

AO  recalls  from  memory  the 
data  about  the  observation, 
calculates  and  orally  reports 
the  results. 

AO  recalls  from  memory 
each  contact  or  important 
new  information  collected 
during  the  search. 

For  each  contact,  he 
should  report  all 
information  gathered,  such 
as  estimated  bearing, 
classification  of  the 
contact,  AOB,  and 
respective  course. 

2.  Critical  Cue  Inventory  (CCI) 

The  CCI  is  a  collection  of  informational  and  perceptual  cues  found  to  be  critical 
cognitive  probes  to  support  decision  (Klein,  Calderwood,  &  MacGregor,  1989).  The  list 
contains  detailed  descriptions  of  the  perceptual  and  judgment  cues  (Hoffman,  Crandall,  & 
Shadbolt,  1998). 

From  the  tree  of  tasks  subordinate  to  the  ART  task,  the  most  important  steps  have 
their  CCI  reproduced  on  a  table.  Predominantly  mechanical  tasks,  such  as  1.1. 2.2.2. 1  (set 
periscope  height)  and  1.1. 2.2. 1.5.1  (set  zoom),  were  not  reproduced  here,  for  they  are 
outside  of  the  scope  of  this  thesis. 
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Table  5.  Critical  Cue  Inventory  for  making  a  visual  search. 


Step:  1.1.2.2.2.3  -  Make  visual  search 

Cue 

Description 

Visual  of  disturbance  on  the 
horizon 

AO  perceives  the  image  of  a  vessel,  aircraft,  or  other 
contact  of  interest  diverging  from  the  sea  and  sky 
visually. 

Periscope  built-in  display 

AO  looks  to  the  relative  and  true  bearing  in  which  the 
contact  was  seen. 

Angle  marks  on  periscope 
reticle 

AO  memorizes  the  angle  covered  by  the  contact  image 
in  the  lens. 

Table  6.  Critical  Cue  Inventory  for  reporting  results. 


Step:  1.1.2.2.4  -  Report  Results 

Cue 

Description 

Memory  of  global  situation 

AO  reports  orally: 

(1)  The  number  of  contacts  sighted. 

(2)  Any  abnormal  condition. 

(3)  Weather  conditions,  if  convenient. 

Memory  of  bearings 

AO  recalls  from  memory  the  bearings  in  which  contacts 
were  sighted,  then  mentally  organizes  them  around  his 
mental  image  of  the  situation. 

The  AO  reports  the  sightings  either  by  true  bearings  or 
by  the  relative  position  from  the  submarine. 

Memory  of  contact  image 
format 

For  each  bearing,  the  AO  recalls  from  memory  the 
contact  sighted.  Evaluating  the  memorized  image,  he 
extracts  all  possible  information: 

(1)  If  close  enough  to  see  the  visual  characteristics  of 
the  contact,  AO  compares  with  the  models  in  his 
memory  and  makes  a  classification  of  the  contact.  If  in 
doubt  he  adds  a  degree  of  uncertainty  (possible, 
probable). 

(2)  From  his  previous  knowledge  of  the  contact  format 
and  size,  AO  compares  the  image  just  seen  and  has  a 
first  estimate  of  the  distance  of  the  contact. 

(3)  If  close  enough  to  see  the  orientation  of  the  contact, 
the  AO  estimates  the  contact’s  AOB,  and  then  calculates 
and  reports  the  contact’s  course  (as  shown  in  Figure  4), 
and  distance  to  route. 

Memory  of  angle  covered  by 
the  contact’s  image 

If  applicable,  the  AO  calculates  mentally  a  more 
accurate  estimate  of  distance  for  the  contact  using  the 
angle  covered  by  the  contact  as  well  as  the  memorized 
information  about  the  height  of  that  contact’s  structure. 
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C.  SELECTION  OF  TASKS 

The  PTT  approach  and  the  insights  gathered  from  the  review  made  in  Chapter  III 
were  used  for  this  thesis.  For  the  selection  of  candidate  tasks  to  be  implemented  for 
training  in  WEBS,  the  following  criteria  were  applied  on  the  total  set  of  tasks  detailed  in 
task  analysis: 

1 .  Absence  of  procedures  or  MOP  cited  solely  in  classified  sources 

2.  Absence  of  procedures  exclusively  related  to  the  operation  of  the  real 
equipment 

3.  Existence  of  complex  cognitive  process  in  the  task  likely  to  be  trained  on  a 
WBS 

4.  Existence  of  pre-required  knowledge  likely  to  be  taught  using  a  WBS 

Following  these  criteria,  the  sub-goals  1.1. 2.2.2. 3  (Make  visual  search)  and 

1.1. 2.2.4  (Report  Results)  were  selected  for  implementation  of  a  demonstration  version  of 
the  software;  the  same  system  was  used  in  the  usability  study. 

D.  MEASURES  AND  STANDARDS  OF  PERFORMANCE 

During  the  execution  of  exercises  on  AT  or  aboard  a  submarine,  the  instructors 
constantly  evaluate  the  performance  of  trainees,  providing  feedback  and  corrections  when 
appropriate.  The  importance  of  feedback  and  tutoring  in  WBLE  was  commented  on  in 
Chapter  III.  In  regard  to  those  aspects,  some  common  MOPs  were  established  for  the 
ARE  task,  with  respective  standards  for  the  100  percent  scoring  (Table  7).  For  this 
implementation,  the  standards  of  performance  are  hypothetical  and  do  not  aim  to  reflect 
actual  measures  of  performance  used  in  the  real  case.  Those  standards  can  be  easily  fixed 
to  match  real  requirements. 
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Table  7.  Measures  of  performance  for  the  All  Round  Look  Part-Task 

Training. 


Indicator 

Description 

Metric 

Standard 

Time  of 
procedure 

Time  taken  from  the 
beginning  of  the  procedure 
until  the  trainee  finishes  the 
horizon  scanning. 

Seconds 

Between  20  and  30 
seconds. 

Periscope  Lens 
elevation 

The  angle  of  elevation  of  the 
periscope  lens  during  the 
movement. 

Proportion 
sea/sky  on 
lens 

2/3  of  sky,  1/3  of 
sea. 

Horizontal 
angle  covered 

If  the  trainee  completes  the 
360  degrees  of  visual 
scanning. 

Degrees 

Greater  than  or 
equal  to  360°. 

Zoom  level 

If  the  trainee  uses  the 
correct  zoom  level 

Magnification 

Equal  to  L5x. 

Detection  of 
contacts 

The  amount  of  visible 
contacts  actually  detected  by 
the  trainee 

Count 

Equal  to  the 
amount  simulated. 

Contact 

classification 

Correct  visual  classification 
of  the  contact 

Angular 

orientation 

The  correct  report  by  the 
trainee  of  the  angular 
localization  of  each  contact 

Angle 

(bearings) 

Equal  to  the  actual 
simulated  bearing, 
plus  or  minus  5°. 

Estimation  of 
distance 

The  estimation  of  distance 
made  by  the  trainee  of  each 
contact 

Yards 

Equal  to  the  actual 
simulated  distance 
plus  or  minus: 

-  100  yards,  for 
contacts  closer  than 
5000  yards. 

-  500  yards  for 
contacts  beyond 

5000  yards. 

AOB 

Correct  estimation  of  the 

AOB 

Degrees 
(180°  for 
starboard  and 
port) 

Equal  to  the  actual 
simulated 
orientation  of  the 
contact,  plus  or 
minus  5°. 
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E.  LEARNING  OBJECTIVES 

Learning  objectives  are  of  paramount  importance  for  the  design  of  any 
instructional  system.  The  established  learning  objectives  should  be  used  to  help  guide  the 
definition  of  requirements  and  development  of  the  instructional  tool.  The  results  of  task 
analysis  identified  the  following  learning  objectives  for  the  ARL  task: 

1 .  Teach  trainees  the  correct  practices  and  the  execution  of  the  ARL  task. 

•  Correct  timing  for  the  horizon  scanning 

•  Correct  lens  elevation  and  setting  the  zoom 

•  Correct  way  to  start  and  finish  a  scanning,  covering  the  360° 

2.  Help  trainees  develop  memorization  skills  to  build  a  mental  model  of  the 
tactical  situation  surrounding  the  submarine. 

3.  Teach  trainees  about  the  cues  needed  for  estimation  of  the  visual  distance 
between  the  contacts. 

4.  Help  the  trainees  recognize  and  classify  contacts  visually. 

5.  Help  trainees  develop  the  ability  to  estimate  the  AOB  of  contacts. 

6.  Teach  the  trainees  the  methods  for  calculation  of  course  of  the  contact  and 
distance  to  route  using  the  estimated  AOB. 

F.  REQUIREMENTS 

Based  on  the  insights  gathered  by  the  task  analysis  performed,  the  following 
requirements  were  established  for  WEBS. 

1.  Purpose 

The  main  purpose  of  WEBS  is  to  provide  a  web-based  simulation  using  a  virtual 
environment  that  represents  the  tasks  normally  executed  by  the  SO  when  using 
periscopes.  WEBS  is  expected  to  help  SOs  develop  necessary  KSA  needed  for  the 
practice  of  operations  on  PD,  preparing  them  for  the  execution  of  missions  of  this  kind 
aboard  the  submarine  or  in  the  simulators. 
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2.  Users  Demographics 

The  primary  users  for  whom  WEBS  is  intended  are  Junior  SOs  in  their  first 
contact  with  this  type  of  mission.  These  users  are  generally  Lieutenants  Junior  Grade, 
male,  with  ages  ranging  from  25  to  30  years,  with  previous  basic  classroom  instruction  in 
the  theoretical  concepts  of  PD  safety  rules. 

The  secondary  users  for  WEBS  are  the  more  experienced  SOs,  who  have  already 
had  some  periscope  training,  and  have  more  solid  knowledge  about  the  periscope 
techniques.  Those  users  are  generally  Lieutenants,  male,  with  ages  ranging  from  28  to  45 
years. 


3.  User  System  Environment 

As  a  web-based  system,  WEBS  must  be  able  to  run  in  most  Internet  browsers,  in 
desktop  and  laptop  personal  computers  from  a  large  range  of  configurations;  the  desire  is 
to  reach  the  capabilities  of  the  average  Internet  user.  The  software  is  not  intended  for  use 
under  an  instructor’s  supervision. 

4.  Limitations 

WEBS  was  not  designed  to  provide  the  following  capabilities: 

•  Mobile  training 

•  Full  mission  training 

•  Extremely  accurate  and  realistic  representation  of  the  sea  surface 

•  Intelligent  Tutoring  System 

•  Accurate  representation  of  periscope  equipment  controls 
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5.  Functional  Requirements 

The  requirements  about  the  speeifie  behaviors  and  functions  expected  from 
WEBS  are: 

1 .  Simulation  of  the  visual  of  a  periscope  screen 

2.  Simulation  of  the  main  information  available  at  a  periscope  screen  for  the 
user 

•  True  bearing  of  the  periscope 

•  Relative  bearing  of  the  periscope 

•  Stadimetric  distance 

•  Crosshairs 

•  Range  divisions  in  minutes  of  arc 

•  Lens  magnification  (zoom) 

3.  Simulation  of  the  horizontal  and  vertical  rotations  of  the  periscope  lens 

4.  Simulation  of  periscope  zooming  in  and  out 

5.  Simulation  of  the  periscope  stadimeter 

6.  Simulation  of  the  sea  surface  and  the  environment 

7.  Simulation  of  common  ships  and  aircrafts  models 

8.  Simulation  of  movements  of  naval  units 

9.  Simulation  of  the  ranging  with  the  stadimeter 

10.  Instructional  system 

1 1 .  Detection  and  evaluation  of  MOP 

12.  Feedback  system 

6.  Non-functional  Requirements 

The  requirements  for  WEBS  in  its  more  general  aspects,  not  related  specifically  to 
its  functionality,  are: 

1 .  Intuitive  user  interface,  allowing  self-learning 

2.  Self-explanatory  setup 

3.  Built-in  instructions  for  quick  familiarization  with  the  user  interface  and 
mission 

4.  Use  only  input  devices  available  in  personal  computers 
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5. 


Run  the  software  via  most  common  web-browsers  to  date  (Google 
Chrome,  Mozilla  Firefox,  Safari,  Internet  Explorer,  and  Opera) 

6.  No  need  for  plugin  software  installations 

G.  CHAPTER  SUMMARY 

The  CTA  performed  for  the  common  missions  of  submarines  on  PD  offered  the 
insights  valuable  for  understanding  the  tasks  performed  by  an  SO  and  adapting  them  for 
the  WBS  training  simulation. 

Once  the  CTA  was  performed,  a  deeper  understanding  of  the  hierarchy  of  sub¬ 
tasks  allowed  the  selection  of  the  best  tasks  to  be  implemented  in  a  WBS.  The  list  of 
perceptual  cues  for  each  task  served  as  a  guide  for  the  definition  of  the  design 
requirements  for  the  simulation.  The  learning  objectives,  an  indispensable  part  of  any 
instructional  system,  were  defined  from  the  knowledge  of  the  hierarchy  of  tasks,  the 
listing  of  critical  cues,  and  knowledge  about  the  overall  task  objectives. 
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V.  PROTOTYPE  DEVELOPMENT 


This  chapter  describes  the  process  of  development  of  WEBS,  based  on  the 
requirements  defined  in  the  previous  chapter.  The  original  intention  of  this  thesis  was  the 
full  development  of  a  WBS  to  help  SOs  training  through  PTT,  to  fill  the  gap  between 
classroom  teaching  and  simulator  training.  However,  most  of  the  tasks  performed  at  a 
periscope  simulator  contain  procedures  and  doctrinal  rules  that  are  classified.  This  fact 
has  limited  the  boundaries  of  what  can  be  implemented  in  the  application  and  released. 
As  explained  in  the  previous  chapter,  the  ART  task  was  selected  to  design  and  develop 
WEBS,  a  sample  WBS  that  could  be  publicly  tested  on  its  usability. 

Although  the  ARE  task  was  the  only  one  submitted  for  usability  testing,  the 
framework  developed  for  it  can  be  used  for  training  of  other  tasks,  just  by  modifying  the 
scenario  settings. 

A.  TECHNOLOGY 

This  section  describes  the  web  technologies  used  for  development  of  WEBS. 
Countless  tools  are  available  nowadays  to  develop  applications  for  the  Internet.  The 
selection  criteria  for  the  technologies  used  in  this  work  were  as  follows: 

•  Open  source:  All  technologies  used  are  open  source  to  date,  and  could  be 
found  and  downloaded,  when  applicable,  from  the  Internet. 

•  No  installation  needed:  For  web  applications,  the  need  for  plugins  or  other 
software  package  installation  can  lead  users  to  frustration,  due  to  their 
inability  to  setup  the  application  or  distrust  of  the  content  of  software 
downloaded  from  the  Internet.  To  avoid  those  caveats,  all  technologies 
depending  on  any  kind  of  installation  were  excluded  from  this  work. 

•  Learning  curve:  The  technologies  used  should  have  an  acceptable  learning 
curve  that  would  allow  a  student  without  previous  experience  in  coding  to 
learn  and  use  them,  within  the  time  frame  of  the  development  of  this 
thesis. 

•  Documentation:  Some  open  source  technologies,  although  having  a 
friendly  learning  curve,  are  not  well  documented,  hindering  the  learning 
process.  The  technologies  used  should  have  enough  documentation  to 
allow  for  learning  using  only  the  documentation  available  on  line. 
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The  following  sub-sections  give  a  further  description  of  the  technologies  used  and 
their  particularities. 

a.  HTML5 

The  standard  used  to  present  content  in  webpages  is  HTML5,  which  stands  for  the 
fifth  version  of  the  Hypertext  Markup  Language  (HTML).  It  is  the  core  technology  used 
to  build  any  web-based  application  meant  to  run  on  a  web-browser.  Some  understanding 
of  the  HTML5  language  is  indispensable  for  the  development  of  webpages.  Due  to  its 
popularity,  sources  from  which  to  learn  HTML5  abound  on  the  Internet,  making  it  easy  to 
learn  without  attending  any  specific  course. 

Among  the  new  features  added  in  the  fifth  version  are  the  canvas  element,  which 
is  especially  important  for  this  thesis.  The  canvas  element  was  introduced  primarily  to 
allow  rendering  of  two-dimensional  (2D)  graphics  and  applications  on  web-browsers,  but 
ended  up  enabling  3D  rendering,  in  association  with  WebGL  (Anyuru,  2012). 

b.  JavaScript 

JavaScript  is  a  prototype-based  scripting  language,  developed  to  leverage  the 
capabilities  of  HTML  by  allowing  the  specification  of  behaviors  through  programming. 
All  modem  web  browsers  include  JavaScript  interpreters,  and  it  is  used  on  the 
overwhelming  majority  of  websites  to  date  (Flanagan,  2011).  JavaScript  is  the  language 
of  choice  to  develop  dynamic  applications  for  the  web.  Due  to  the  relevance  that  the 
language  has  gained  in  the  Internet,  since  the  first  release  of  the  language,  successive 
advances  have  been  made  to  the  standard  and  to  the  interpreters,  making  it  faster,  lighter, 
and  more  powerful.  Nowadays  JavaScript  has  left  behind  its  scripting-language  roots  and 
has  become  an  efficient,  general  purpose  language  that  supports  multiple  programming 
paradigms. 


c.  Cascading  Style  Sheets  (CSS) 

The  Cascading  Style  Sheets  (CSS)  is  a  language  used  to  define  the  appearance 
and  format  of  webpages  written  in  HTML5.  Although  not  strictly  necessary  for  the 
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styling  of  webpages,  the  use  of  CSS  makes  the  design  of  the  graphical  user  interface 
(GUI)  for  webpages  much  easier  and  richer. 

d.  WebGL 

From  all  the  technologies  listed  here,  WebGL  is  perhaps  the  most  important 
regarding  the  development  of  a  WBS,  because  it  is  the  standard  that  makes  3D  rendering 
possible  on  an  HTML5  canvas.  WebGL  enables  hardware-accelerated  3D  graphics  to  be 
included  in  a  web  browser.  It  is  an  open  standard  that  can  be  implemented  and  used 
without  charge  for  royalties.  It  runs  natively  in  the  web  browsers  that  offer  support  for  it, 
without  the  need  for  any  plugin  or  installation  (Anyuru,  2012). 

The  first  version  of  the  standard  was  released  in  2011  (Khronos  Group,  2011), 
having  the  second  version  released  in  2013.  It  is  a  relatively  new  technology,  which  still 
lacks  support  from  some  web  browsers.  Figure  12  offers  a  summary  table  of  the  support 
offered  to  WebGL  by  the  time  that  this  thesis  was  written.  The  table  shows  the  support 
per  version  of  the  web  browsers.  The  current  versions  are  outlined  in  black,  and  the  table 
also  shows  a  preview  of  support  in  the  next  versions.  One  can  note  from  the  table  that  the 
support  for  mobile  devices  is  currently  very  low. 


Figure  12.  Cross  browser  support  for  WebGL  as  of  August  2014  (from  Deveria, 

2014). 


Besides  the  need  for  compatibility  with  the  browser,  there  are  additional 
compatibility  issues  with  the  hardware,  most  frequently  with  older  control  processor  units 
(CPUs).  Also,  some  hardware,  although  natively  compatible,  comes  with  WebGL 
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disabled  by  the  factory  settings.  The  known  hardware  with  this  condition  is  listed  in 
Table  8,  based  on  the  “blacklist”  maintained  by  the  Khronos  Group  (2014).  Safari  is  the 
only  modem  desktop  browser  that  comes  with  WebGL  disabled  by  default. 


Table  8.  List  of  hardware  with  WebGL  disabled  by  factory  settings. 


Browser 

Operating 

System 

Hardware 

Mozilla 

Firefox 

Windows  older 
than  XP 

All 

Windows  newer 
than  XP 

NVIDIA  <257.21 

ATI/AMD  <  10.6 

Intel  drivers  earlier  than  September  2010 

Mac  OS  X<  10.6 

All 

Google 

Chrome 

All 

Intel  Mobile  945  Express  family  of  chipsets 

NVDIA  GeForce  FX  Go5200 

ATI  FireNV  2400 

Parallels  drivers  older  than  version  7 

S3  Trio  cards 

Windows 

All  graphics  drivers  before  2009 

Windows  XP 

All  ATEAMD  drivers  older  than  version  10.6 

All  NVIDIA  drivers  older  than  version  257.21 

All  Intel  drivers  older  than  version  14.42.7.5294 

Mac  OS  X 

ATI  Radeon  XI 900 

NVIDIA  GeForce  7300  GT 

All  cards  have  WebGL’s  initializing  disabled 

Linux 

ATE  AMD  GPUs  with  proprietary  AMD  drivers 
older  than  version  8.98 

NVIDIA  GPUs  with  proprietary  NVIDIA  drivers 
older  than  version  295 

Multisampling  is  disabled  on  Intel  IvyBridge 
cards 

NVIDIA  Quadro  FX  1500 

WebGL  is  disabled  on  the  dynamically  switching 
NVIDIA+Intel  GPUs 

Android 

WebGL  is  disabled  on  devices  that  do  not 
support  ARB  robustness  or  EXT  robustness  with 
context  reset  notification 

The  lack  of  wide  cross-browser  and  cross-platform  support  is  an  important  issue 
when  considering  a  technology  that  could  be  used  to  build  a  WBS.  One  of  the  critical 
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points  of  web  development  is  that  the  developer  has  no  control  over  the  user’s  computer 
configuration.  The  applications  that  fail  to  reach  and  be  adopted  by  a  wide  range  of  users 
can  be  sentenced  to  oblivion. 

Another  important  aspect  is  the  computational  power  of  the  user’s  machine.  Even 
when  the  user  has  all  the  software  and  hardware  ready  to  run  the  3D  application  but  the 
computer  processor  is  designed  for  lighter  tasks,  it  may  happen  that  the  application  will 
not  run,  or  it  will  provide  an  unsatisfactory  user  experience. 

Even  though  the  current  incompatibilities  of  WebGL  may  represent  a  problem  for 
widespread  applications,  the  usefulness  of  the  technology  already  surpasses  its  caveats. 
Furthermore,  it  may  be  just  a  matter  of  time  before  WebGL  is  running  without  problem  in 
all  browsers  and  platforms,  just  like  JavaScript  is  running  now. 

e.  JavaScript  Libraries 

All  JavaScript  libraries  used  in  this  effort  are  free  and  distributed  under  the  MIT 
license.  The  library  that  had  the  most  important  role  in  the  development  was  “three.js,” 
which  is  a  JavaScript  library  and  API  that  automates  several  routine  procedures  of 
WebGL,  making  it  easier  to  code  3D  applications. 

The  library  jQuery  was  used  for  help  on  JavaScript  coding.  It  was  used  in 
association  with  the  collection  of  widgets  called  “jQuery  UI,”  which  offers  a  variety  of 
user  interface  elements  implemented  with  CSS  in  combination  with  jQuery.  The 
collection  of  widgets  includes  buttons,  sliders,  and  different  types  of  menus  among  other 
elements  valuable  for  the  building  of  a  user  interface. 

Finally,  the  Tweenjs  library  was  used  to  automate  the  interpolation  and  use  of 
animations. 

/.  Blender 

Blender  is  powerful  open  source  3D  modeling  software  that  is  distributed  under 
the  GNU  General  Public  License  (GNU  GPL)  version  two.  It  provides  not  only 
modeling,  but  also  texturing,  UV  mapping,  and  animation  capabilities,  among  others.  The 
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3D  models  and  scenes  developed  in  Blender  can  easily  be  exported  to  the  JavaScript 
Object  Notation  (JSON)  format  allowing  easy  integration  with  the  web  browser. 


g.  Gimp 

The  GNU  Image  Manipulation  Program  (GIMP)  was  the  selected  program  to 
create  and  modify  most  of  the  images  and  textures  used  in  this  project.  It  is  no-cost  image 
manipulation  software  distributed  under  the  GNU  GPL  version  three.  GIMP  offers  a 
powerful  and  resourceful  tool  that  is  well  documented  and  easy  to  learn. 

B.  MODELS 

This  section  describes  the  models  built  for  the  implementation  of  the  WEBS 
prototype.  Since  none  of  the  technology  tools  used  in  this  effort  were  designed  to  be  a 
game  engine,  most  of  the  functions  provided  by  these  engines  needed  to  be  coded  from 
scratch. 


1.  Periscope  Screen 

A  user  who  is  looking  at  the  eyepiece  does  not  see  the  periscope  itself,  only  an 
image  that  could  be  seen  through  the  periscope  lens.  The  periscope  was  implemented  as 
an  empty  object  3D  to  which  the  camera  was  added  in  the  scene  graph.  The  typical 
circular  cropping  of  the  image  of  the  periscope  was  implemented  using  an  overlapping 
canvas  layer,  in  which  the  information  of  the  display  could  also  be  rendered,  as  shown  in 
Figure  13.  The  information  set  provided  by  the  display  consists  of  the  level  of 
magnification,  stadimetric  distance  measured,  and  current  true  and  relative  bearing  of  the 
periscope. 

The  periscope  object  has  six  degrees  of  freedom;  as  such  it  is  possible  to  rotate 
around  and  translate  it  on  three  axes.  Since  in  this  version  of  WEBS  the  user  does  not 
need  to  command  the  periscope  to  rise  or  to  be  lowered,  the  only  control  allowed  to  the 
user  is  the  rotation  of  the  periscope  object  around  the  vertical  axis.  The  other  movements 
cannot  be  controlled  by  the  user  interaction,  which  is  reserved  for  the  simulation  of  the 
rocking  of  the  submarine  transmitted  to  the  periscope. 
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The  camera  object  acts  as  the  periscope  lens,  and  it  has  only  one  degree  of 
freedom — it  could  be  rotated  around  its  own  horizontal  axis,  simulating  the  elevation  of 
the  lens.  As  it  is  subordinated  to  the  periscope,  it  will  also  follow  all  the  periscope’s 
rotation  to  the  left  and  right. 


Figure  13.  The  periscope  screen  display. 


The  alternation  of  the  levels  of  magnification  was  simulated  in  three.js  just  by 
altering  the  camera  field  of  view,  and  then  forcing  it  to  recalculate  and  update  the 
perspective  projection. 

One  important  feature  of  optical  periscopes  is  the  stadimeter.  Even  though  it  is 
generally  not  used  in  the  ARE  mission,  this  functionality  should  be  simulated  to  complete 
the  basic  framework  that  allows  rapid  prototyping  of  other  missions.  The  periscope 
stadimeter  estimates  the  distance  of  an  object  of  known  height  by  measuring  the  angle 
between  the  bottom  and  the  top  of  the  object.  To  measure  the  angle,  the  user  moves  a 
mirrored  image  of  the  object  until  the  bottom  of  the  mirrored  image  matches  the  top  of 
the  real  image.  This  effect  was  simulated  simply  by  rendering  the  scene  image  as  a 
texture  on  a  semi-transparent  plane.  The  plane  is  positioned  in  front  of  the  camera,  and  its 
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elevation  is  controlled  by  the  user,  who  acts  on  the  assigned  stadimeter  control.  The 
elevation  of  the  plane  is  then  converted  into  an  angle,  and  the  distance  is  calculated  and 
displayed  on  the  screen,  as  shown  in  Figure  14. 


Figure  14.  Simulation  of  the  image  lagging  of  a  periscope  stadimeter. 

2.  Environment  Model 

The  ocean  model  was  based  on  a  three.js  shader  for  realistic  water;  it  was 
implemented  by  Bouny  (2014).  The  model  uses  reflection,  refraction,  bump  mapping, 
and  normal  mapping  to  simulate  the  aspect  of  the  water  surface  on  a  geometry. 
Additionally,  the  vertices  of  the  geometry  can  have  their  positions  dynamically  updated 
as  a  function  of  time,  which  enables  simulation  of  the  effect  produced  by  the  waves. 

The  surrounding  sky  is  simulated  with  the  use  of  lights  and  a  cubic  skybox;  a 
texture  needed  for  a  particular  situation  is  applied  on  that  skybox.  The  skybox  images 
used  in  this  system  were  shared  on  the  Internet  by  Reijerse  (2006)  under  a  Creative 
Commons  license. 


72 


3.  Ships  and  Aircrafts  Models 

A  Java  style  of  object-oriented  programming  was  emulated  in  JavaScript  as 
proposed  by  Flanagan  (2011)  to  enable  an  organized  definition  of  classes,  objects,  and 
inheritance.  A  generic  class  “Ship”  was  created  to  represent  most  of  the  attributes  and 
methods  common  to  both  ship  and  aircraft  objects.  This  class  defined  methods  for  the 
animation  of  the  object  and  helped  translate  the  state  of  the  object  into  a  format  common 
to  the  naval  domain,  such  as  bearing,  course,  heading,  etc.  The  3D  model  was  added  as 
an  attribute  of  an  object.  When  a  new  object  is  instantiated,  the  3D  model  associated  with 
it  is  automatically  loaded  and  added  to  the  scene,  with  the  attitude  defined  in  its 
constructor. 

The  three.js  API  allows  the  use  of  several  formats  of  3D  models.  The  models  used 
in  this  work  were  in  the  JSON  and  OBJ-MTL  formats.  Most  of  the  3D  models  were 
downloaded  originally  from  the  Delta  3D  asset  library.  The  library  contains  some  3D 
objects  in  the  proprietary  format  for  Autodesk  3DS  Max.  Those  models  were  converted 
to  the  JSON  format  using  a  converter  written  in  Python,  provided  with  the  three.js  API 
package. 


C.  GRAPHICAL  USER  INTERFACE  (GUI) 

A  GUI  that  was  designed  to  facilitate  the  user  interaction  with  the  application  was 
created  using  j  Query  UI.  Most  of  the  GUI  elements  were  used  on  the  main  menu,  where 
the  user  could  set  the  parameters  of  the  simulation  before  the  simulation  began  running. 
A  snapshot  of  the  main  menu  aspect  is  shown  in  Figure  15. 
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Figure  15.  WEBS  main  menu. 


The  user  can  interact  with  the  GUI  elements,  such  as  buttons,  menu  items,  etc.,  by 
clicking  with  the  mouse,  or  using  the  keyboard,  by  highlighting  the  element  with  the 
“tab”  key  and  then  selecting  it  with  “enter”  or  “return.”  Affordances  were  included  in  the 
form  of  tooltips  to  add  extra  information  for  the  user  on  how  to  interact  with  the  GUI. 
The  tooltip  that  provides  information  about  an  element  of  the  interface  is  triggered  and 
shown  when  the  user  hovers  above  that  element  with  the  mouse  cursor  without  actually 
selecting  it. 

A  bilingual  capability  was  added  to  the  application.  The  user  could  select  between 
English  or  Brazilian  Portuguese  in  the  main  menu.  Consequently,  all  the  dialogs,  menus, 
and  instructions  of  the  interface  were  presented  in  the  selected  idiom.  Using  the  HTML5 
local  storage  capability,  once  the  user  selects  an  idiom,  this  selection  is  recorded  and 
automatically  set  for  the  next  time  the  user  accesses  the  application. 

The  main  menu  offers  a  selection  between  several  missions  available  on  a  sidebar 
on  the  left,  after  which  the  content  of  the  central  portion  of  the  menu  gets  switched  to  the 
configuration  panel  of  that  mission.  For  the  ARE  training  mission,  only  two  settings  were 
made  available:  the  number  of  contacts  and  the  environment.  The  user  could  select  the 
number  of  contacts  either  by  setting  a  specific  number  from  one  to  ten,  or  by  selecting 
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any  range  between  one  and  ten.  If  the  user  selected  a  range,  a  random  number  of  contacts 
was  generated  between  the  minimum  and  maximum  values  of  that  set. 

When  the  user  clicks  on  the  button  “Start  Simulation,”  the  scenario  gets  loaded 
and  the  simulation  starts,  changing  the  GUI  to  the  simulation  state.  Most  of  the  user 
inputs  in  the  simulation  state  are  made  through  the  keyboard.  Table  9  summarizes  the 
controls  used  for  this  mission.  The  objective  was  to  use  controls  that  are  as  intuitive  as 
possible,  and  reuse  standard  keys  to  avoid  user  confusion. 


Table  9.  List  of  keyboard  controls  used  in  the  ARL  mission. 


Key 

Action 

Left  and  Right  arrows 

Rotate  the  periscope 

Up  and  Down  arrows 

Change  the  elevation  of  the  lens 

Move  the  stadimeter  image  up  and  down  in 
mode  stadimeter 

“z” 

Toggle  through  the  available 
magnifications 

“s” 

Set  the  object’s  height  for  stadimeter 
calculations 

“d” 

Toggle  stadimeter  mode  on  and  off 

“Shift” 

Increase  the  turning  speed  of  the  periscope 

Toggle  the  stadimeter  from  fine  to  coarse 
adjusting 

“p” 

Pause  the  simulation 

If  the  user  wanted  to  take  the  stadimeter  distance,  he  could  use  either  the  key 
commands  or  the  mouse  wheel,  instead  of  the  up  and  down  arrows. 

1.  Scoring  and  User  Feedback 

During  the  execution  of  the  mission,  all  MOPs  established  in  Table  7  for  the  ARL 
task  are  evaluated.  After  all  assets  are  loaded  and  the  simulation  is  ready  to  run,  the 
mission  begins  with  the  simulation  paused.  The  trainee  initially  cannot  visualize  the 
scene  until  he  or  she  clicks  a  button  to  start  running  the  time.  In  the  initial  state,  the 
periscope  is  pointing  to  the  000“  relative  bearing,  with  the  elevation  at  zero.  During  the 
execution  time,  the  application  detects  the  following: 
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•  Total  time  of  execution  of  the  task 

•  Elevation  of  the  periscope 

•  Whether  or  not  the  360°  were  covered 

•  If  the  trainee  used  zoom 

After  the  ARE  is  finished,  the  trainee  is  asked  to  declare  the  following  data: 

•  Number  of  contacts  detected 

•  Classification  of  contacts 

•  Estimated  bearing  on  which  the  contact  was  detected 

•  Estimated  distance  of  each  contact 

•  Estimated  angle  on  the  bow  for  each  contact 

At  the  end,  the  trainee  is  presented  with  a  feedback  table,  showing  his  answers 
and  the  answers  provided  by  the  system  (the  ‘objective’  truth). 

D.  CHAPTER  SUMMARY 

This  chapter  discussed  the  main  aspects  of  the  development  of  the  WEBS.  The 
WEBS  application  was  developed  with  functionalities  that  can  be  extended  and  adapted 
to  implement  other  missions  and  PTT. 

Knowing  the  technology  used  in  this  effort  and  its  characteristics,  we  discussed 
possible  limitations  of  the  application.  The  main  limitations  existing  to  date  are  probably 
the  few  hardware  and  software  incompatibilities  of  WebGL.  Applications  with  3D 
rendering  for  the  web  could  be  easily  developed  with  the  open-source  tools  presented; 
however,  the  extent  of  the  distribution  and  the  user’s  experience  with  the  application  can 
be  impacted  by  those  same  limitations  of  hardware  and  programming  environments. 
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VI.  USABILITY  STUDY 


One  major  objective  has  been  established  for  the  usability  study — ^to  provide  the 
answers  to  the  research  questions  4  and  5: 

Question  4.  What  are  the  usability  issues  of  a  web-based  3D  simulation 
(efficiency,  effectiveness,  and  user  satisfaction  with  the  user 
interface)? 

Question  5.  Do  WebGL  compatible  web-browsers  have  the  resources  to  fully 
support  a  web-based  3D  simulation? 

To  address  these  questions,  a  modified  version  of  WEBS  was  used  in  a  usability 
study  that  engaged  human  subjects  (users)  recruited  among  the  Naval  Postgraduate 
School  (NPS)  body  of  students.  Because  the  study  involved  human  subjects,  including 
the  collection  of  information  from  them,  all  procedures  and  documents  related  to  the 
study  had  to  be  reviewed  and  approved  by  the  Institutional  Review  Board  (IRB)  team 
from  NPS  prior  to  execution  of  the  study.  The  details  about  the  design,  execution,  and 
results  of  this  study  are  discussed  in  this  chapter. 

A.  DESIGN  OF  THE  EXPERIMENT 

A  starting  set  of  conditions  was  established  for  the  study.  To  ensure  that  the 
results  would  reflect  the  real  issues  with  web-based  applications,  WEBS  was  hosted  as  a 
webpage  on  the  Internet;  this  ensured  that  the  application  was  subjected  to  the  same 
advantages  and  caveats  discussed  in  Chapter  III.  Also  noteworthy  is  the  fact  that  the 
participants  received  only  a  link  along  with  the  recruitment  email  (full  text  in  Appendix 
A),  and  accessed  the  application  using  their  personal  computers.  Being  that  the  aim  was 
to  address  the  compatibility  issues  cited  in  Chapter  V,  no  restriction  was  made  regarding 
the  machine  configuration  in  the  recruitment  script. 

To  preserve  a  sense  of  self-learning  experience  and  expose  possible  usability 
weaknesses,  all  instructions  on  how  to  execute  the  study  were  provided  via  the 
application  itself — there  was  no  personal  contact  between  the  researchers  and  the 
participants.  The  following  sections  present  further  details  about  the  elements  of  this 
usability  study. 
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1. 


The  Modified  Mission 


A  modified  mission  was  configured  to  serve  the  usability  study.  Because  the  main 
objective  of  the  study  was  not  to  test  the  instructional  effectiveness  of  the  application  but 
the  usability  of  the  user  interface,  this  modified  mission  was  focused  on  presenting  the 
participants  with  the  functionalities  of  WEBS  and  not  necessarily  on  training  techniques. 
The  mission  presented  to  the  participants  consisted  of  a  tutorial  about  the  functionalities 
of  WEBS,  followed  by  an  ARE  task. 

After  the  user  accessed  the  link  provided  in  the  recruitment  email,  he/she  was 
redirected  to  a  webpage  with  the  survey  consent  text  (full  text  in  Appendix  B).  The 
consent  briefly  described  the  objectives  of  the  study  and  how  the  study  would  be 
conducted.  In  this  page,  a  link  was  provided  to  a  version  of  the  consent  form  in  a  print 
friendly  format,  in  case  the  participant  wanted  to  print  it  for  future  reference.  Clicking  on 
the  button  entitled  “I  consent  to  participate,”  the  participant  was  redirected  to  the  page 
with  WEBS  main  menu.  This  button  also  triggered  a  function  to  save  a  key  in  the  local 
storage  of  the  user’s  browser.  Before  loading,  the  main  menu  webpage  would  check  that 
key,  and  if  the  key  was  not  found,  the  participant  would  be  redirected  to  the  consent  page. 
Besides  the  fact  that  the  link  to  the  main  menu  was  not  provided,  this  procedure  was 
added  to  assure  that  no  one  would  accidentally  have  access  to  the  study  without  having 
consented  to  their  participation  in  the  study. 

After  checking  for  consent,  the  main  menu  page  would  proceed  with  a  series  of 
checks  during  the  loading  process;  the  procedural  diagram  of  the  checks  is  reproduced  in 
Figure  16.  The  checks  aimed  to  assure  the  following  conditions: 

1.  The  instructions  would  be  presented  in  the  selected  language  from  the 
very  beginning. 

2.  The  window  of  the  user’s  browser  was  maximized. 

3.  The  user  had  a  browser  and  hardware  compatible  with  WebGL. 

4.  In  case  the  user’s  browser  was  not  Google  Chrome,  the  user  would  be 
advised  to  use  it.  This  advice  was  based  on  the  fact  that  in  preliminary 
tests  performed  while  preparing  for  the  main  study,  WEBS  presented  the 
best  performance  in  the  Google  Chrome  browser. 
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5. 


If  the  user  had  already  done  the  full  tutorial,  he  or  she  would  be  given  the 
option  to  be  redirected  to  the  survey  or  to  execute  the  tutorial  again.  This 
was  made  to  assure  that,  in  case  the  user  accidentally  reloads  the  page 
after  finishing  the  tutorial,  he  or  she  would  not  be  forced  to  execute  the 
full  tutorial  mission  again. 


Figure  16.  Routine  of  preload  checks  performed  by  WEBS. 


Later,  after  the  survey  results  were  analyzed,  it  was  revealed  that  these  checks 
should  have  included  the  instructions  to  enable  WebGL  in  Safari  browsers. 
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The  main  menu  page  was  modified  from  the  original  version  to  have  only  two 
submenus:  1)  the  tutorial  mission  and  2)  a  menu  with  further  explanation  about  the 
project  and  credits  to  the  third  party  libraries  used  in  the  development.  Both  submenus 
had  a  button  to  start  the  tutorial. 

During  the  tutorial,  the  user  was  asked  to  interact  with  the  application  using  one 
system  control  at  the  time.  The  goal  of  the  tutorial  was  to  teach  the  user  all  system 
controls,  and  to  prevent  the  user  from  getting  lost  or  confused  during  the  session  when 
one  control  was  introduced  and  some  other  controls  were  blocked.  In  other  parts  of  the 
tutorial,  the  application  would  detect  that  the  user  was  acting  differently  than  expected  for 
the  task  and  present  additional  clues  to  help  the  learning  process. 

Figure  18  shows  an  example  of  the  tutorial  task  presented  to  the  users  during  the 
tutorial  session.  The  successful  execution  of  the  tutorial  task  would  automatically  trigger 
the  next  task.  After  completion  of  the  tutorial  session  dedicated  to  learning  the 
application  controls,  the  user  received  the  final  task  that  consisted  of  performing  an  ART. 


Idiom 

PortuguSs  English 

MissionB 

Tutorial 

All  RCwr,  :  LooV^ 

Looking  Inter.  lIs 
GoOEx 

attack  .  .i  .s*op 

About  this  Project 


This  is  the  simplified  version  of  an  on-line  3D  simulation,  for  usability  assessment. 

Follow  the  Instructions  presented  by  the  Tutor.  At  the  end  you  will  be  redirected  to  the  survey  about  your  experience 
using  it. 

Click  the  "Start  Tutorial"  button  below  to  begin. 


Start 

Tutorial 


Figure  17.  The  main  menu  of  the  modified  mission. 
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The  user  would  receive  a  brief  instruction  about  the  way  an  ARL  task  should  be 
performed  and  the  information  that  the  system  would  not  be  assessing  his  or  her 
performance.  The  end  of  the  task  was  triggered  either  after  30  seconds  or  when  the  user 
covered  the  360  degrees  with  the  periscope.  After  that,  a  table  with  all  information  that 
could  be  gathered  from  the  ARL  as  well  as  the  common  errors  were  shown  for  the  user’s 
information.  An  example  of  such  a  table  is  shown  in  Figure  19. 


Tutorial  -  Task  3  of  4  x 

Excellentll 

Task  3  -  Zooming: 

Each  time  you  press  "z"  key  on  your  keyboard,  the  periscope  zoom  is  changed.  Your  task 
now  is  to  explore  the  available  magnifications  and  then  set  it  back  to  the  minimum  zoom 
(1.5X). 

Notice  that  the  level  of  magnification  is  showed  on  the  upper-left  comer  of  the  periscope  screen  (like 
shown  In  the  image  below).  This  periscope  has  3  levels  of  magnification:  l.Sx,  6.0x  and  12.0x; 

Click  "Ok”  to  continue. 


Figure  18.  Example  of  tutorial  task. 
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End  of  Tutorial 


When  you  close  this  dialog  box,  you  will  be  redirected  to  the  survey. 


1  Common  Mistakes  I 

1  Information  that  a  trained  submariner  should  know  after  this  All  Round  Look:  | 

Image 

Classification 

Estimated 

Bearing 

Estimated 

Distance 

Estimated 

Course 

Passenger  Liner 

060° 

8000  yards 

000° 

Time  taken 

00  :  24 
seconds 

Reversion  of  turn? 

ASW  Helicopter 

210° 

2800  yards 

320° 

Periscope  too  high? 

Periscope  too  low? 

Amphibious  Ship 
(LPD) 

290° 

8400  yards 

B 

Fall  to  complete  360° 
turn? 

Oil  Platform 

100° 

13800 

yards 

XXX 

Used  zoom? 

Go  to  Survey 


Figure  19.  Feedback  table  presented  at  the  end  of  ARL  task. 


After  closing  the  feedback  table  or  clicking  on  “Go  to  Survey”  button,  the  user 
survey  would  open  in  a  new  window  or  tab  of  the  user’s  browser. 

2.  Participant  Recruitment 

Participation  in  this  study  was  strictly  voluntary  and  anonymous.  The  only 
exclusion  criteria  applied  was  that  the  participants  should  be  adults  (over  20  years  of 
age),  have  higher  education,  be  male  or  female,  and  be  within  the  NPS  body  of  students 
or  faculty. 

After  approval  of  the  process  by  the  IRB,  a  recruitment  email  (full  text  available 
in  Appendix  A)  was  sent  to  the  whole  body  of  students.  The  survey  was  answered  by  a 
total  of  78  participants;  only  72  completed  all  questions  in  the  survey. 
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3.  Post-Session  Survey 

The  survey  (available  in  Appendix  C)  was  created  using  LimeSurvey,  and  made 
available  on  the  Internet.  LimeSurvey  is  an  online  open  source  survey  tool  (Schmitz, 
2012);  the  version  used  in  this  study  was  hosted  on  the  NPS  server  and  made  available  to 
NPS  students  and  faculty.  The  tool  complied  with  all  required  standards  of  protection  of 
personal  data.  The  main  body  of  the  survey  was  composed  of  50  questions  divided  into 
four  sections:  (1)  Demographics,  (2)  User  Experience,  (3)  Learning  Experience,  and  (4) 
Comments  and  Suggestions.  No  question  was  made  mandatory,  so  that  the  participants 
could  skip  any  question  they  judged  as  not  applicable.  This  particular  configuration  was 
made  mainly  to  allow  the  collection  of  data  from  participants  who  could  not  use  WEBS 
due  to  some  unforeseen  technical  problems. 

The  first  section,  Demographics,  was  aimed  at  gathering  information  about  the 
participants  and  their  habits  using  the  Internet.  The  following  section,  User  Experience, 
had  questions  regarding  the  experiences  the  participants  just  had  with  WEBS.  In  the 
Learning  Experience  section,  the  questions  focused  on  the  experience  and  opinions  of  the 
participants  about  learning  tools  like  WBLE  and  about  other  simulations.  In  the  last 
section,  Comments  and  Suggestions,  a  set  of  free  open-ended  questions  were  made  to 
allow  the  participants  to  add  comments  or  suggestions  for  issues  that  perhaps  were  not 
listed  in  any  other  section. 

LimeSurvey  supports  internal  logic  between  questions,  which  facilitates  the 
assignment  of  questions  relevant  to  the  participant  on  an  individual  basis.  For  example,  a 
participant  who  had  never  experienced  simulations  before  would  not  be  asked  about  his 
or  her  previous  experience  with  simulations.  Also,  the  same  question  could  be  submitted 
to  two  different  classes  of  respondents  and  achieve  individualized  results  related  to  each 
class. 

The  data  collected  by  the  survey  tool  could  be  extracted  in  different  formats,  such 
as  spreadsheets  (.csv  or  Microsoft  Excel),  PDF,  or  SPSS,  allowing  for  easy  analysis. 
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B. 


RESULTS  ANALYSIS 


All  the  responses  to  the  survey  are  summarized  in  a  graphical  form  in  Appendix 
G;  this  section  illustrates  the  analysis  of  a  selected  set  of  responses. 

1.  Participants’  Background 

From  all  the  respondents,  12.5%  have  used  a  full-scale  periscope  simulator 
before.  A  slightly  greater  number  of  participants,  16.67%  of  them,  had  previous 
experience  with  games  featuring  periscope  simulations.  Other  types  of  computer-based 
simulations  (not  including  the  submarine  domain)  were  used  by  48.28%  of  the 
participants.  From  the  respondents  of  this  survey,  only  19.44%  were  submariners.  Among 
the  submariners,  35.71%  had  less  than  3  years  of  service  in  submarines.  The  techniques 
presented  in  this  thesis  are  based  on  procedures  taught  at  BNSS,  and  common  to  most 
conventional  (non-nuclear)  submarines.  The  submariners  who  participated  did  not 
necessarily  have  the  same  background  or  are  familiar  with  those  techniques.  Given  the 
low  percentage  of  submariners,  it  becomes  inappropriate  to  make  general  conclusions  and 
associate  them  with  the  profession  of  submariners.  Only  one  of  the  submariners  had 
experience  as  an  instructor  in  submarine  courses;  he  had  two  years’  experience  in  that 
capacity. 

2.  Online  Habits 

A  selected  set  of  questions  were  aimed  at  collecting  the  data  about  the  familiarity 

of  the  participants  with  Internet  and  web-based  applications.  The  results  showed  that  the 

participants  were  quite  familiar  with  the  use  of  the  Internet  and  online  applications.  Only 

one  participant  claimed  to  access  the  Internet  once  a  week,  on  average,  and  the  vast 

majority  (98.61%)  claimed  to  access  the  Internet  on  a  daily  basis.  All  the  participants 

were  accustomed  to  Internet  browsing  and  using  email.  The  third  most  frequently  used 

web  applications  were  the  ones  associated  with  e-commerce  (95.83%),  followed  by 

multimedia  streaming  (87.5%),  and  social  networks  (73.61%).  The  attendance  in  online 

courses  came  next  with  66.67%,  ahead  of  online  games  (26.39%).  To  check  if  the 

participants  had  a  different  idea  about  online  courses  and  online  training,  it  was  also 

asked  if  they  had  ever  done  any  kind  of  web-based  training  before,  not  giving  a  definition 
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of  the  concept  of  web-based  training.  Instead  of  a  discrete  answer,  this  question  required 
an  answer  using  the  Likert  scale  with  five  levels  to  capture  if  the  participants  felt  that 
they  had  been  enrolled  online  training  previously.  The  results  were  higher  than  for  online 
courses,  having  those  who  agreed  and  those  who  strongly  agreed  at  the  level  of  85.71%. 

Relevant  insights  came  from  the  hardware  that  people  used  to  access  the  Internet 
and  how  they  used  it.  Among  the  surveyed  population,  the  device  of  choice  to  access  the 
Internet  was  the  personal  computer.  Laptop  and  desktop  users  represented  80.55%  of  the 
participants  who  use  the  Internet.  This  is  important  information  for  all  simulations  based 
on  WebGL,  especially  knowing  its  limited  compatibility  with  mobile  devices.  Only 
5.56%  of  the  interviewed  participants  preferred  tablets,  while  smartphones  were  the 
devices  of  choice  for  13.89%  of  study  participants.  It  is  generally  considered  that  not  all 
tasks  can  be  trained  with  the  same  effectiveness  on  different  devices — the  acquisition  of 
some  types  of  knowledge  and  skills  are  better  suited  for  one  type  of  device,  and  another 
type  of  knowledge  and  skills  is  better  suited  for  another  type  of  device.  However,  when 
asked  what  device  they  would  choose  to  perform  a  web-based  training,  57.14%  of  the 
participants  chose  the  laptop,  21.43%  preferred  a  desktop  computer,  12.86%  the  tablet, 
and  only  5.71%  of  participants  chose  the  smartphone. 

Participants  were  also  asked  about  the  devices  that  they  owned  for  their  personal 
use,  excluding  those  that  they  currently  own  but  which  are  destined  for  the  use  by  another 
member  of  the  family  (for  example,  tablets  being  predominantly  used  by  the  participant’s 
children).  The  most  commonly  owned  devices  were  the  laptop  (97.22%)  and  the 
smartphone  (90.28%).  Although  tablets  were  not  so  popular  for  Internet  access,  they 
came  in  third  place  with  63.89%.  The  desktops  came  next,  with  only  50%.  This  is 
important  data  for  the  design  of  future  applications,  since  the  input  devices  generally 
differ  a  bit  between  desktops  and  laptops.  Most  of  the  laptops  do  not  use  a  mouse,  and 
instead  they  offer  touchpads.  Given  this  understanding,  it  would  not  be  a  good  choice  to 
bind  a  control  of  the  application  strictly  to  the  mouse  wheel,  for  example. 
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3.  User  Experience  with  WEBS 

Most  questions  about  the  user  experience  with  WEBS  were  made  using  the  Likert 
scale.  To  facilitate  the  visualization  of  the  results,  the  graph  in  Figure  20  was  created  with 
the  mean  values  for  each  question,  considering  only  the  valid  answers.  The  Likert  values 
correspondence  range  from  1  for  a  “strongly  disagree”  answer  to  5  for  a  “strongly  agree” 
answer.  The  graph  provides  a  good  idea  about  the  average  response;  however,  as  it  is 
based  on  mean  values,  important  trends  could  be  masked.  A  more  complete 
understanding  is  achieved  by  checking  the  distribution  plot  and  the  standard  deviation  for 
each  question.  The  complete  set  of  responses  can  be  seen  in  Appendix  G. 


Strongly 

strongly 

dfsagree 

agree 

The  web- based  simulation  provided  enough  guidance  - 1  was  able  to 
use  it  without  additional  help 

The  quality  of  graphics  used  in  web-based  simulation  was  good. 

The  textual  information  and  numbers  were  easy  to  read  and 
provided  necessary  inforination  to  accomplish  the  task. 

The  controls  were  easy  to  use. 
The  use  of  controls  was  intuitive. 

The  buttons  {elements  of  user  interface)  were  of  appropriate  sire. 

The  choke  of  colors  in  user  Interface  made  the  instructions  easy  to 
read. 

The  meaning  of  menu  items  was  clear. 
The  purpose  of  this  web-based  simulation  was  clear. 

I  was  able  to  complete  the  task  (execute  the  mission)  very  efflcientiy. 

This  web-based  simulation  reminded  of  my  experience  In  using  a  real 
periscope. 

I  did  not  experience  any  technical  difficulties  with  this  web- based 
simulation. 

Technical  difficulties  did  not  influence  negatively  the  use  of  this  web- 
based  simulat'on. 
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Figure  20.  Summary  of  Likert  scale  mean  values  for  the  user  experience  with 

WEBS. 
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The  overall  picture  gives  the  idea  that  the  elements  of  the  interface  were  on  a 
satisfactory  level,  suggesting  also  that  there  was  room  for  improvement.  A  slightly  lower 
satisfaction  was  expressed  regarding  the  feeling  of  self-efficacy  of  the  participants — this 
was  the  question  that  inquired  about  their  ability  to  complete  the  task.  The  evaluation  of 
open-ended  questions  suggests  that  this  may  have  been  caused  by  the  fact  that,  although 
the  application  did  not  evaluate  the  participants,  it  offered  feedback  about  what  the 
performance  standards  of  a  trained  individual  should  be.  In  some  comments,  the 
participants  would  say  that  they  felt  that  if  they  had  trained  a  little  bit  more  they  would 
have  achieved  a  higher  level  of  performance.  Another  aspect  to  be  revised  is  the 
explanation  about  the  purpose  of  the  WBS.  The  results  show  that  about  one-third 
(31.94%)  of  participants  was  in  doubt  or  neutral  about  the  purpose  of  the  simulation. 

To  get  some  idea  about  the  realism  of  the  simulation,  only  the  submariners  were 
asked  about  the  extent  to  which  WEBS  reminded  them  of  their  experience  with  a  real 
periscope.  The  average  Likert  scale  renders  a  fair  result  of  3.1,  suggesting  that  when  it 
came  to  simulated  level  of  realism,  there  was  a  lot  of  room  for  improvement.  Again,  it  is 
important  to  be  reminded  that  the  small  group  of  submariners  surveyed  in  this  study  may 
have  been  composed  of  participants  with  a  diverse  background,  such  as  individuals  who 
have  been  used  to  different  types  of  equipment.  In  open-ended  comments,  three 
submariners  expressed  their  concern  about  the  fact  that  the  controls  were  too  different 
from  what  they  were  used  to.  Another  participant  commented  about  his  or  her  concern 
that  the  bio-mechanical  part  of  the  task  could  not  be  reproduced  on  a  personal  computer. 

Finally,  although  most  of  the  participants  claimed  not  to  have  had  major  technical 
difficulties,  like  image  freezing,  flickering,  etc.,  a  great  number  of  them  (40.28%)  felt 
that  some  technical  difficulty  influenced  negatively  their  experience  with  the  application, 
while  30.56%  were  neutral  on  this  point. 

4.  The  Gap  Classroom  -  Simulator  -  Operational  Environment 

One  of  the  research  issues  addressed  by  this  thesis  was  the  assumption  of  an 
existing  gap  between  the  theoretical  classroom  teaching  and  the  hands-on  training  on 
simulators  or  actual  operational  environments  (equipment).  The  respondents  with 
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experience  in  training  with  simulators  were  asked  about  the  way  they  felt  regarding  their 
preparedness  before  and  after  the  training  session  on  a  full-scale  simulator.  Three  main 
aspects  were  addressed  about  how  they  felt:  (1)  if  they  felt  prepared  to  begin  the 
simulator  training;  (2)  if  they  had  a  clear  understanding  of  the  tasks  that  would  be 
performed  at  the  simulator  facility;  and  (3)  if  the  amount  of  training  on  the  simulator  that 
they  used  was  enough  to  prepare  them  for  the  operational  environment.  The  summary  of 
mean  Likert  values  for  these  responses  can  be  seen  in  Figure  21 . 


Strongly 

disagree 


The  number  of  hours  I  had  in  a  full  scale  simulator 
was  enough  to  understand  the  theory  and  master  the 
necessary  skills  that  I  needed. 


Before  training  with  a  full  scale  simulator,  I  had  clear 
understanding  of  all  the  tasks  I  was  expected  to 
execute. 


Before  starting  a  training  with  a  full  scale  simulator  I 
felt  well  prepared  to  take  part  in  that  training 
session. 
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Figure  2 1 .  Summary  of  Likert  scale  mean  values  for  the  user  self-reported 
preparedness  regarding  simulator  training. 


Considering  the  mean  values  in  those  questions,  the  results  give  the  idea  that  there 
is  a  little  gap  in  training,  either  before  or  after  attending  simulator  facilities  at  military 
training  centers.  However,  once  again  we  need  to  understand  that  the  mean  values  can  be 
misleading.  Considering  only  the  respondents  who  marked  less  than  or  equal  to  three, 
(they  were  neutral  or  they  disagreed  with  the  statement)  the  following  could  be 
concluded: 
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•  For  the  statement  that  the  amount  of  training  in  simulators  was  enough  to 
prepare  them  for  the  real  operational  environment,  34.3%  of  the 
participants  answered  that  they  disagreed  or  were  neutral. 

•  For  the  statement  that  before  beginning  the  training  the  trainees  had  a 
complete  understanding  of  the  tasks  that  would  be  performed,  36.1%  of 
the  participants  answered  that  they  disagreed  or  were  neutral. 

•  For  the  statement  that  they  felt  well  prepared  to  take  part  in  simulator 
trainings,  41.7%  of  the  participants  answered  that  they  disagreed  or  were 
neutral. 

These  results  show  that,  although  not  unanimous,  there  is  an  unfulfilled  gap  in 
training  for  some  people.  The  fact  that  this  gap  is  not  generalized,  reinforces  the  potential 
for  the  use  of  WBS  made  available  to  the  trainees  on  a  voluntary  basis;  the  trainees  who 
felt  a  need  for  more  training  between  the  classroom  and  training  simulators,  and  between 
the  training  on  simulators  and  the  real  operational  environment  could  opt  to  fdl  that  need 
by  using  the  type  of  training  tools  like  WBS. 

5.  Acceptance  of  Web-Based  Training 

A  set  of  questions  was  focused  on  the  willingness  of  the  participants  to 
voluntarily  participate  in  online  training.  Overall  the  results  show  that  the  participants 
had  a  positive  attitude  towards  web-based  training. 

The  participants  with  experience  on  training  simulators  were  asked  if  they  would 
use  a  WBS  to  prepare  for  an  upcoming  training  on  the  full-scale  simulator,  and  83.78% 
of  them  agreed  that  they  would.  The  only  submariner  with  experience  as  an  instructor 
agreed  that  this  tool  would  be  valuable  as  an  instructional  help.  When  asked  about  the 
place  of  preference  where  they  would  perform  web-based  training,  62.86%  responded 
that  they  would  do  it  at  home,  while  35.71%  would  prefer  doing  it  at  work.  As  for  the 
point  in  time  when  they  would  do  the  training  with  a  WBS,  most  people  (58.57%)  had 
opinion  that  it  would  be  useful  both  before  and  after  the  training  with  the  full-scale 
simulator;  a  total  of  35.71%  of  participants  found  it  to  be  more  useful  before  in-base 
training  with  simulators,  and  2.86%  found  it  not  to  be  useful  at  all. 
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Figure  22  offers  a  summary  of  the  mean  results  of  questions  related  to 
participants’  willingness  to  use  and  their  acceptance  of  web-based  training. 


Strongly 

disagree 


If  I  had  a  choice,  I  would  replace  some  hours  of  traditional  classroom 
lectures  for  web-based  classes  that  I  could  access  from  home. 


This  type  of  three-dimensional  (3D)  web-based  simulations  could  be  a 
valuable  tool  for  training. 


Web-based  three  dimensional  (3D)  simulation  is  a  valuable  tool  for 
training. 


If  I  had  access  to  web-based  simulation  that  I  could  use  from  home,  I 
would  use  it  to  improve  my  performance. 


This  web-based  simulation  would  be  a  good  tool  for  training  of 
periscope  handling,  and  it  could  be  used  to  help  improve  these  type  of 
skills. 


Hands-on  training  motivates  me. 
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Figure  22.  Summary  of  Likert  scale  mean  values  for  the  user’s  propensity  to 

adopt  web-based  trainings. 


The  majority  of  the  participants  (90%)  feel  motivated  by  hands-on  training.  Most 
people  (71.43%)  agreed  that  they  would  voluntarily  replace  a  part  of  their  classroom 
lecture  time  with  the  web-based  content  that  could  be  accessed  from  home.  Most 
participants  (88.57%)  also  had  the  opinion  that  WBS  and  applications  like  WEBS 
represented  a  valuable  tool  for  training.  More  specifically,  participants  suggested  that 
WEBS  would  be  a  valuable  tool  for  training  of  periscope  handling  (the  mean  value  of 
3.8).  The  participants  also  showed  a  willingness  to  voluntarily  use  a  WBS  from  home  to 
improve  their  performances  with  the  same  mean. 
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6.  Open-Ended  Questions 

There  were  three  open-ended  questions,  inviting  the  participants  to  volunteer  any 
additional  comments,  concerns,  or  suggestions  for  improvement  that  they  might  have. 
Adding  up  the  responses  for  all  three  questions,  a  total  of  3 1  participants  provided  their 
comments.  From  those,  14  reported  technical  difficulties  that  either  impeded  or  hampered 
the  execution  of  the  tasks  proposed.  However,  some  of  these  limitations  could  have  been 
avoided  by  adding  the  instructions  about  how  to  enable  WebGL  in  the  Safari  browser,  as 
the  application  failed  to  do  it  (the  same  was  also  reported  by  participants  who  were  Safari 
users). 

The  second  more  frequently  cited  concern,  mostly  provided  by  the  submariners 
and  pilots,  was  related  to  the  lack  of  simulation  of  mechanical  movements  of  the  body, 
and  the  difference  between  the  human-machine  interfaces  on  personal  computers 
compared  to  the  real  environment.  This  concern  reinforced  the  suitability  of  a  PTT 
approach  for  WBS,  focusing  only  on  those  tasks  not  strictly  linked  to  the  operation  of 
equipment  and  mechanical  skills.  In  addition,  it  stresses  the  need  for  providing  plenty  of 
information  to  the  users,  to  avoid  them  being  unaware  of  the  objectives  of  the  application. 

Less  frequent  but  still  noteworthy  were  the  suggestions  to  improve  the  color 
patterns  of  the  simulation,  using  different  keys  to  make  the  interface  more  intuitive,  and 
improving  the  realism  by  adding  more  environmental  cues.  Also,  four  participants 
expressed  their  concerns  about  the  clarity  of  instructions  provided  in  the  application, 
suggesting  that  further  improvement  should  be  made  to  make  them  clear. 

7.  General  Analysis 

The  analysis  of  the  results  was  positive  regarding  the  usefulness  and  suitability  of 
WBS  for  distance  learning  and  training.  The  background  of  the  population  that 
participated  in  the  study  revealed  high  familiarity  with  web-based  applications  and  a  high 
willingness  for  the  adoption  of  this  type  of  training  on  a  voluntary  basis.  The  results  also 
contribute  towards  the  assumption  that  there  exists  an  instructional  gap  between  the 
classroom  teaching  and  the  training  on  simulators,  at  least  for  a  portion  of  the  trainees. 
The  use  of  WBS  could  become  a  valuable  tool  in  reducing  this  gap,  ensuring  that  the 
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trainees  went  to  training  centers  with  a  higher  understanding  and  mastery  of  the  basic 
skills. 


The  technical  difficulties  related  to  the  technology  employed  for  this  application 
were  also  made  evident.  Although  a  large  majority  of  the  participants  was  able  to  execute 
the  application,  there  was  still  a  great  number  of  participants  who  were  not  able  to  take 
total  advantage  of  simulations  built  using  this  technology.  It  was  also  made  evident  that, 
although  well  accepted,  the  application  has  room  for  several  improvements  before  being 
finally  distributed  for  real  training. 
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VII.  CONCLUSIONS  AND  FUTURE  WORK 


A.  CONCLUSIONS 

Computer-based  simulations  have  been  used  successfully  to  aid  the  training  of 
complex  skills  and  dangerous  tasks  common  in  military  professions.  Their  use,  however, 
requires  the  trainees  to  attend  training  sessions  in  specific  locations  inside  a  training 
facility  that  typically  has  limited  availability.  Very  specifically,  for  the  training  of 
periscope  depth  safety  rules  for  the  submariners,  using  the  “eyes-only”  style  common  to 
conventional  submarines,  the  trainees  need  to  master  several  concepts  and  skills  before 
their  very  first  contact  with  the  periscope  simulator.  The  theoretical  concepts  are  taught  in 
the  classroom  environment,  but  the  mastery  of  skills  demands  individual  effort  and  takes 
time  not  available  during  the  lecture  portion  of  a  course.  Frequently  the  trainees  use  part 
of  their  training  time  in  the  simulator  to  correct  and  master  those  basic  skills,  wasting 
precious  time  that  could  be  used  for  more  advanced  training. 

Web-based  simulations  can  potentially  be  used  to  diminish  this  gap  between 
theory  and  practice,  providing  an  instructional  environment  that  the  trainees  can  access 
from  home  and  adapt  to  their  time  and  convenience.  Today,  advances  in  web-based 
technology  allow  the  delivery  of  simulations  that  include  complex  virtual  environments 
without  the  need  for  installation  of  plugins  or  the  use  of  proprietary  software. 

For  this  thesis,  a  prototype  of  a  web-based  periscope  simulation  was  developed  to 
allow  the  testing  of  this  concept.  A  usability  study  was  performed  in  which  the  volunteers 
accessed  the  online  prototype,  accomplished  a  tutorial  session  and  a  simple  mission,  then 
responded  to  an  online  survey,  and  left  their  impressions  about  the  prototype  and  this 
training  modality.  The  results  showed  that  people  were  receptive  to  web-based  training; 
most  of  the  military  with  experience  of  training  with  simulators  were  willing  to  use  a 
part-task  training  web-based  simulation  to  improve  their  performance. 

Technical  issues  were  also  evidenced.  Most  of  the  participants  were  able  to  use 
the  application  as  intended,  but  some  participants  experienced  difficulties  due  to 
incompatibility  of  their  hardware  or  software  with  the  technologies  used  in  WBS,  or  due 
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to  the  lack  of  computational  power.  This  study  confirmed  our  belief  that  web-based 
simulations  for  training  should  not  be  extremely  dependent  on  the  hardware 
configuration;  they  should  not  lose  the  most  remarkable  advantage  of  web-based 
applications:  the  ability  to  deliver  the  same  content  on  different  platforms  without  the 
need  for  software  installation,  and  to  deliver  this  content  to  the  user  virtually  anywhere 
and  anytime.  But  web-based  technology  is  constantly  and  rapidly  evolving.  Each  day 
more  computational  power  is  available  at  the  user’s  home.  This  leads  to  the  conclusion 
that  the  web-based  simulations  represent  valuable  and  viable  tools  for  improving  future 
training,  making  a  link  between  the  training  on  the  full-scale  simulators  and  the 
classroom  environment,  as  well  as  being  useful  for  the  refreshing  of  skills  for  those 
professionals  who  are  already  trained. 

B.  FUTURE  WORK 

The  prototype  application  built  for  this  thesis  had  the  usability  of  the  user 
interface  as  its  objective.  Further  development  of  the  application  should  include: 

•  Creation  of  additional  missions  to  enable  training  of  other  important  skills. 

•  Development  of  simulations  for  part-task  training  on  other  subjects  in  the 
submarine  domain  not  related  to  the  periscope. 

•  Creation  of  a  library  of  relevant  3D  models,  more  realistic  and  optimized 
for  use  in  the  Internet. 

•  Implementation  of  an  intelligent  tutoring  system  to  leverage  the  self-paced 
learning. 

•  Improvements  in  the  GUI  and  in  the  usability  of  the  application. 

•  Optimization  of  the  code  to  improve  the  user  experience  while  demanding 
fewer  hardware  resources. 

•  Development  of  better  graphical  effects  to  improve  the  level  of  realism. 

•  Development  of  a  software  framework,  along  the  lines  of  a  game  engine, 
to  ease  future  development  of  simulations. 

Beyond  further  development  of  the  application,  a  deeper  usability  study  should  be 
conducted  with  a  complete  web-based  simulation  aimed  only  at  the  final  target 
population  (that  is,  ‘end  users’).  Also  extremely  important  is  the  research  focused  on 
training  effectiveness  and  transfer  of  skills  using  this  type  of  training  tool. 
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The  submarine  operations  domain  was  the  basis  for  this  thesis,  but  research 
should  also  be  done  to  extend  this  concept  to  other  domains. 


95 


THIS  PAGE  INTENTIONALLY  LEFT  BLANK 


96 


APPENDIX  A.  SURVEY  RECRUITMENT  SCRIPT 


To  All  NFS  Students, 

You  are  invited  to  participate  in  research  study  that  will  help  us  test  the 
elements  of  a  Web-Based  Submarine  Periscope  Simulator;  you  will  be 
able  to  run  it  on  your  own  computer  using  a  web  browser  of  your  choice. 

You  will  go  through  following  steps:  (1)  do  a  simple  5  minute-long 
training  session,  (2)  execute  the  main  task,  (3)  answer  a  short  survey  about 
your  experience. 

Your  participation  is  voluntary,  anonymous,  and  will  only  take  about  15 
minutes.  ^  personal  data  are  collected. 

Please  click  here  to  participate  in  the  study.  If  the  link  does  not  work, 
please  copy  and  paste  this  link  into  your  browser: 
http://www.simulasub.org 

If  you  have  questions  regarding  the  research,  contact  Principal 
Investigator,  Dr.  Amela  Sadagic,  831-656-3819,  asadagic@nps.edu.  If 
you  have  any  questions  regarding  your  rights  as  a  research  subject,  please 
contact  the  Naval  Postgraduate  School  IRB  Chair,  Dr.  Larry  Shattuck, 
831-656-2473,  lgshattu@nps.edu. 

We  recognize  how  busy  you  are.  Your  participation  supports  the  effort 
of  a  thesis  student,  and  we  appreciate  your  willingness  to  assist  in  this 
short  study. 

If  you  have  any  questions,  please  do  not  hesitate  to  contact  us. 

Sincerely, 

Amela  Sadagic,  Ph.D. 

Research  Associate  Professor 

MOVES  Institute 

Naval  Postgraduate  School 
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APPENDIX  B.  SURVEY  CONSENT 


Naval  Postgraduate  School 
Consent  to  Participate  in  Anonymous  Survey 

You  are  invited  to  partieipate  in  a  researeh  study  entitled  ‘‘User  Aeeeptanee  Survey  of  Web-Based 
Submarine  Periseope  Simulator.”  The  purpose  of  this  researeh  is  to  help  in  evaluating  the 
potential  aeeeptanee  of  a  web-based  periseope  simulator  for  submarine  offieers  basie  training. 
Your  partieipation  will  be  a  valuable  souree  of  information  to  improve  future  researeh  in  this 
field. 

The  experiment  eonsists  of  eompleting  the  online  tutorial  about  how  to  use  the  simulator,  and 
then  eompleting  the  first  mission,  an  all-around  seareh  using  the  periseope  simulator.  The 
eompletion  of  these  two  tasks  will  take  about  10  minutes.  After  this  you  will  be  redireeted  to  a 
survey  about  your  experienee  with  the  web-based  simulation.  This  survey  should  take  about  five 
minutes  to  eomplete.  Your  partieipation  is  voluntary.  There  is  no  direet  benefit  to  you  for 
partieipating.  If  you  partieipate,  you  are  free  to  skip  any  questions  or  stop  partieipating  at  anytime 
without  penalty.  Your  responses  are  anonymous.  Results  of  the  survey  will  be  used  responsibly 
and  proteeted  against  release  to  unauthorized  persons.  All  data  eolleeted  will  be  safeguarded  on 
the  NFS  seeure  server;  however,  there  is  a  minor  risk  that  data  eolleeted  eould  be  mismanaged. 

If  you  have  questions  regarding  the  researeh,  or  experienee  any  injury,  eontaet  Prineipal 
Investigator,  Dr.  Amela  Sadagie,  asadagie@nps.edu.  If  you  have  any  questions  regarding  your 
rights  as  a  researeh  subjeet,  please  eontaet  the  Naval  Postgraduate  Sehool  IRB  Chair,  Dr.  Larry 
Shattuek,  831-656-2473,  lgshattu@nps.edu. 


I  |l  eonsent  to  partieipate. 
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APPENDIX  C.  SURVEY  QUESTIONS 


Besides  the  whole  set  of  questions,  the  logic  applied  is  also  reproduced  in  this 
appendix.  Some  questions  were  repeated,  so  that,  using  the  logic,  they  could  be  targeted 
at  different  respondents. 


Usability  assessment  of  the  Web-Based 
Submarine  Periscope  Simulator 

This  survey  aims  to  assess  the  usability  of  a  web-based  tridimensional  simulation  for  training. 

There  are  50  questions  in  this  survey 

Demographics 


Are  you  a  submarine  officer? 

Please  choose  only  one  of  the  following: 

O  Yes 
O  No 


How  many  years  of  experience  in  the  submarine  service? 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '1  [D1]'  ( Are  you  a  submarine  officer?  ) 

Please  choose  all  that  apply: 

CH  Less  than  1  year 
CH  Between  1  and  3  years 
[U  Between  3  and  6  years 
r~l  More  than  6  years 


Have  you  ever  used  a  full  scale  periscope  simulator  for  your  training? 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '1  [D1]'  ( Are  you  a  submarine  officer?  ) 

Please  choose  only  one  of  the  following: 

O  Yes 
O  No 
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How  much  preparation  (in  hours)  do  you  typically  do  before  starting  a 
training  session  with  a  full  scale  simulator? 

Only  answer  this  question  if  the  following  conditions  are  met; 

Answer  was  'Yes’  at  question  '3  [D3]'  (Have  you  ever  used  a  full  scale  periscope  simulator  for  your  training?  ) 
Only  numbers  may  be  entered  in  this  field. 

Please  write  your  answer  here: 


Please  answer  an  estimated  amount  of  hours. 

"Preparation"  means  that  you  had  to  study  and/or  train  by  yourself  before  going  to  the  simulator  session. 


If  you  had  access  to  web-based  (on-line)  simulation,  would  you  use  it  to  help 
you  prepare  for  your  possible  upcoming  training  on  the  full  scale  simulator  in 
base? 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '3  [D3]'  (Have  you  ever  used  a  full  scale  periscope  simulator  for  your  training?  ) 
Please  choose  only  one  of  the  following; 

O  Yes 
O  No 


Have  you  ever  been  an  instructor  for  submarine  courses? 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '1  [D1]'  ( Are  you  a  submarine  officer?  ) 

Please  choose  only  one  of  the  following: 

O  Yes 
O  No 


For  how  many  years  have  you  been  an  instructor? 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '6  [D6]'  (Have  you  ever  been  an  instructor  for  submarine  courses?) 
Only  numbers  may  be  entered  in  this  field. 

Please  write  your  answer  here: 
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Would  you  use  an  online  simulation  like  this  to  execute  a  training  session  as 
an  instructor? 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '6  [D6]'  (Have  you  ever  been  an  instructor  for  submarine  courses?) 

Please  choose  only  one  of  the  following: 

O  Yes 
O  No 


Have  you  ever  played  any  submarine  game  with  a  periscope  simulation? 

Please  choose  only  one  of  the  following: 

O  Yes 
O  No 


Have  you  ever  used  a  computer-supported  simulation  for  training? 
(examples:  flight  simulation  or  training  simulations  for  ground  operations). 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'No'  at  question  '1  [D1]'  ( Are  you  a  submarine  officer?  ) 

Please  choose  only  one  of  the  following: 

O  Yes 
O  No 


How  much  preparation  do  you  typically  do  before  starting  a  training  session 
with  those  simulations? 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '10  [DIO]'  (Have  you  ever  used  a  computer-supported  simulation  for  training? 
(examples:  flight  simulation  or  training  simulations  for  ground  operations).) 

Only  numbers  may  be  entered  In  this  field. 

Please  write  your  answer  here: 


Please  answer  an  estimated  amount  of  hours. 

"Preparation"  means  that  you  had  to  study  and/or  train  by  yourself  before  going  to  the  simulator  session. 
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If  you  had  access  to  web-based  (on-line)  simulation,  would  you  use  it  to  help 
you  prepare  for  your  possible  upcoming  training  on  the  fuil  scale  simulator? 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '10  [DIO]'  (Have  you  ever  used  a  computer-supported  simulation  for  training? 
(examples;  flight  simulation  or  training  simulations  for  ground  operations).) 

Please  choose  only  one  of  the  following: 

O  Yes 
O  No 


Select  all  applications  that  you  use  on  regular  basis  in  the  Internet  (seiect  ali 
that  apply): 

Please  choose  all  that  apply; 

□  Browse  the  network  (use  web  browser) 

□  On-line  courses  or  distance  learning  modules 

□  On-line  shopping,  e-commerce  or  e-banking 

□  E-mail 

□  Multimedia  streaming  (online  movies,  music,  etc.) 

□  Social  networks 

□  Online  games 

□  I  participate  in  wikis  and  web  forums 

□  I  maintain  a  Blog 

□  I  maintain  a  website 

r~]  Other: 


With  what  frequency  do  you  access  internet  for  any  appiication  mentioned  in 
the  previous  question? 

Please  choose  only  one  of  the  following: 

O  Daily 
O  Weekly 
O  Monthly 

O  I  rarely  use  the  Internet 
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What  is  the  device  you  use  the  most  to  access  the  Internet? 

Please  choose  only  one  of  the  following: 

O  Desktop  computer 
O  Laptop  computer 
O  Tablet 
O  Smartphone 
O  Game  console 

O  Other 


Mark  below  all  devices  that  you  own  for  your  personal  use: 

Please  choose  all  that  apply: 

O  Desktop  computer 
n  Laptop  computer 

□  Tablet 

□  Smartphone 

□  Game  console 

EH  Other;  | 

Mark  only  those  that  are  for  your  own  use. 
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User  Experience 


About  your  experience  with  the  web-based  simulation. 

The  web-based  simulation  provided  enough  guidance  -  I  was  able  to  use  it 
without  additional  help 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


The  quality  of  graphics  used  in  web-based  simuiation  was  good. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


The  textual  information  and  numbers  were  easy  to  read  and  provided 
necessary  information  to  accomplish  the  task. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 
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The  controls  were  easy  to  use  (zooming  in/out,  moving  left/ right  and 
up/down,  etc.]- 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


The  use  of  controls  was  intuitive. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


The  buttons  (elements  of  user  interface)  were  of  appropriate  size. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


The  choice  of  colors  in  user  interface  made  the  text,  numbers  and  other 
instructions  easy  to  read. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 
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The  meaning  of  menu  items  was  clear. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


The  purpose  of  this  web-based  simulation  was  clear. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


I  was  able  to  complete  the  task  (execute  the  mission)  very  efficiently. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


This  type  of  three-dimensional  (3D)  web-based  simulations  could  be  a 
valuable  tool  for  training. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 
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This  web-based  simulation  reminded  of  my  experience  in  using  a  real 
periscope. 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '1  [D1]'  ( Are  you  a  submarine  officer?  ) 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


I  did  not  experience  any  technical  difficulties  with  this  web-based  simulation 
(such  as:  image  was  freezing,  flickering,  being  too  slow,  etc.) 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


There  were  technical  difficulties,  but  they  did  not  influence  negatively  the  use 
of  this  web-based  simulation. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 
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This  web-based  simulation  would  be  a  good  tool  for  training  of  periscope 
handling,  and  it  could  be  used  to  help  improve  these  type  of  skills. 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '1  [D1]'  ( Are  you  a  submarine  officer?  ) 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


Do  you  have  any  concern  about  the  use  of  this  product? 

Please  choose  only  one  of  the  following: 

O  Yes 
O  No 


What  are  they? 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '32  [U16]'  (Do  you  have  any  concern  about  the  use  of  this  product?) 
Please  write  your  answer  here: 
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Learning  Experience 

About  the  way  you  learn. 

Hands-on  training  motivates  me. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


In  my  past  experience,  before  I  would  start  training  with  a  full  scale 
simulator  I  felt  well  prepared  to  take  part  in  that  training  session. 

Only  answer  this  question  if  the  foiiowing  conditions  are  met: 

Answer  was  'Yes'  at  question  '3  [D3]'  (Have  you  ever  used  a  full  scale  periscope  simulator  for  your  training?  ) 
Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


In  my  past  experience,  before  I  would  start  training  with  a  full  scale 
simulator  I  felt  well  prepared  to  take  part  in  that  training  session. 

Only  answer  this  question  if  the  foiiowing  conditions  are  met: 

Answer  was  'Yes'  at  question  '10  [DIO]'  (Have  you  ever  used  a  computer-supported  simulation  for  training? 
(examples:  flight  simulation  or  training  simulations  for  ground  operations).) 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 
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In  my  past  experience,  before  I  would  start  training  with  a  full  scale 
simulator,  1  had  clear  understanding  of  all  the  tasks  I  was  expected  to 
execute. 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '3  [D3]'  (Have  you  ever  used  a  full  scale  periscope  simulator  for  your  training?  ) 
Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


In  my  past  experience,  before  I  would  start  training  with  a  full  scale 
simulator,  I  had  clear  understanding  of  all  the  tasks  I  was  expected  to 
execute. 

Only  answer  this  question  if  the  following  conditions  are  met; 

Answer  was  'Yes'  at  question  '10  [DIO]'  (Have  you  ever  used  a  computer-supported  simulation  for  training? 
(examples:  flight  simulation  or  training  simulations  for  ground  operations).) 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


The  number  of  hours  I  had  in  a  full  scale  simulator  was  enough  to  understand 
the  theory  and  master  the  necessary  skills  I  needed  before  going  to  the 
related  operational  environment. 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '3  [D3]'  (Have  you  ever  used  a  full  scale  periscope  simulator  for  your  training?  ) 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 
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The  number  of  hours  1  had  in  a  full  scale  simulator  was  enough  to  understand 
the  theory  and  master  the  necessary  skills  I  needed  before  going  to  the 
related  operational  environment. 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '10  [DIO]'  (Have  you  ever  used  a  computer-supported  simulation  for  training? 
(examples:  flight  simulation  or  training  simulations  for  ground  operations).) 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


If  I  had  access  to  web-based  simulation  that  I  could  use  from  home,  I  would  use  it 
to  improve  my  performance. 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '3  [D3]'  (Have  you  ever  used  a  full  scale  periscope  simulator  for  your  training?  ) 
Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


If  I  had  access  to  web-based  simulation  that  I  could  use  from  home,  I  would  use  it 
to  improve  my  performance. 

Only  answer  this  question  if  the  following  conditions  are  met: 

Answer  was  'Yes'  at  question  '10  [DIO]'  (Have  you  ever  used  a  computer-supported  simulation  for  training? 
(examples:  flight  simulation  or  training  simulations  for  ground  operations).) 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


113 


Web-based  three  dimensional  (3D)  simulation  is  a  valuable  tool  for  training. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


If  I  had  a  choice,  I  would  replace  some  hours  of  traditional  classroom 
lectures  for  web-based  classes  that  I  could  access  from  home. 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 


Have  you  ever  done  any  kind  of  web-based  training  before? 

Please  choose  only  one  of  the  following: 

O  Strongly  disagree 
O  Disagree 

O  Neither  agree  nor  disagree 
O  Agree 
O  Strongly  Agree 
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If  some  web-based  training  is  found  equally  effective  on  all  those  devices, 
what  kind  of  device  would  you  prefer  to  use  for  your  web-based  training  in 
the  future? 

Please  choose  only  one  of  the  following: 

O  Desktop  computer 
O  Laptop  computer 
O  Tablet 
O  Smartphone 

O  Other 


Regarding  web-based  (online)  training,  I  would  prefer  doing  it: 

Please  choose  only  one  of  the  following: 

O  At  home 
O  At  work 
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Suggestions  and  Opinions 


When  do  you  think  a  web-based  simulation  would  be  more  useful  as  a 
training  tool? 

Please  choose  only  one  of  the  following: 

O  Before  using  a  full  scale  simulator,  to  help  better  visualize  and  understand  the  theory. 

O  After  using  a  full  scale  simulator,  to  help  retain  knowledge  and  skills. 

O  Both  before  and  after  training  with  a  full  scale  simulator. 

O  I  do  not  think  it  would  be  useful  at  all. 

O  Other 


Do  you  have  any  suggestion  on  how  to  improve  the  web-based  simulation  as 
a  training  tool? 

Please  write  your  answer  here: 


Provide  any  comment  about  your  experience  with  the  web-based  simulation. 

Please  write  your  answer  here: 
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APPENDIX  D.  SURVEY  RESULTS 


In  this  appendix  the  responses  to  the  survey  are  summarized.  In  the  graphs,  the 
numbers  in  parenthesis  report  the  quantity  of  people  who  selected  that  option.  The 
questions  that  used  the  Likert  scale  have  the  following  options  represented  in  the  graphics 
in  a  numeric  format: 

1 .  Strongly  disagree 

2.  Disagree 

3.  Neither  agree  nor  disagree 

4.  Agree 

5.  Strongly  agree 


Table  10.  Summary  charts  of  survey  results. 


Are  you  a  submarine  officer? 


■gD36To' 


19.44% 

_ 1 

^ _ 1 

Yes  (14)  No  (58) 


How  many  years  of  experience  in 
the  submarine  service? 

■  <lyear(3)  II>1&<3(2) 

□  >3&<6(7)  H>6(2) 


50.00% 


21.43% 

1  A  IQO/r. 

14  29% 

1 

1 

y 

IL 
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Have  you  ever  used  a  full  scale 
periscope  simulator  for  your 
training? 


64^9% 


How  much  preparation  (in  hours)  do  you 

typically  do  before  starting  a  training 

session  with  a  full  scale  simulator? 

Count 

9 

Sum 

4 

Standard  deviation 

0.5 

Average 

0.44 

Minimum 

0 

1st  quartile  (Ql) 

0 

2nd  quartile  (Median) 

0 

3rd  quartile  (Q3) 

1 

Maximum 

1 

Would  you  use  web-based 
simulation  to  prepare  for  an 
upcoming  training  in  a  full-scale 
simulator  in  base? 


83.78% 


16.22% 

Yes  (31)  No  (6) 


Have  you  ever  been  an  instructor 
for  submarine  courses? 


92.86.^ 


7.14% 

Yes (1) 

No  (13) 
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For  how  many  years  have  you 
been  an  instructor? 

Count 

1 

Sum 

2 

Standard 

deviation 

0 

Average 

2 

Minimum 

2 

Maximum 

2 

Would  you  use  an  online 
simulation  like  this  to  execute  a 
training  session  as  an  instructor? 


100.00 


% 

0.00% 

Yes  (1)  No  (0) 


Have  you  ever  played  any 
submarine  game  with  a  periscope 
simulation? 


83.33% 


16.67% 

Yes  (12)  No  (60) 


Have  you  ever  used  a  computer- 
supported  simulation  for  training? 


Yes  (28)  No  (30) 
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How  much  preparation  do  you 
typically  do  before  starting  a 
training  session  with  those 
simulations? 


Count 

28 

Sum 

42.75 

Standard 

deviation 

1.7 

Average 

1.53 

Minimum 

0 

1st  quartile  (Q1) 

0.625 

2nd  quartile 
(Median) 

1 

3rd  quartile  (Q3) 

2 

Maximum 

8 
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What  is  the  device  you  use  the  most  to  access  the  Internet? 


Desktop  Laptop  Tablet  (4)  Smartphone  Game  console  Other  (0) 

computer  (16)  computer  (42)  (10)  (0) 


Mark  below  all  devices  that  you  own  for  your  personal  use: 


Desktop  Laptop  Tablet  (46)  Smartphone  (65)  Game  console  Other  (3) 

computer  (36)  computer  (70)  (24) 
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With  what  frequency  do  you  access 
Internet  for  any  application 
mentioned  in  the  previous 
question? 

“98.61% -  - 


1.39%  0.00%  0.00%  0.00% 


Daily  (71)  Weekly  Monthly  I  rarely  No 
(1)  (0)  use  the  answer 

Internet 
(0) 


The  quality  of  graphics  used  in 
web-based  simulation  was  good. 


-  “52:78% 


The  web-based  simulation  provided 
enough  guidance  - 1  was  able  to  use 
it  without  additional  help 


51.39% 


26.39% 

5.56%  4  i7o/„ 

4  17% 

1(4)  2(3)  3(6)  4(19)  5(37)  No 

answer 

(3) 


The  textual  information  and 
numbers  were  easy  to  read  and 
provided  necessary  information  to 
accomplish  the  task. 


1(2)  2(1)  3(9)  4(34)  5(23)  No 

answer 

(3) 
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The  controls  were  easy  to  use. 


1(1)  2(2)  3(5)  4(24)  5(35)  No 

answer 

(5) 


The  buttons  (elements  of  user 
interface)  were  of  appropriate  size. 


1(0)  2(1)  3(9)  4(34)  5(25)  No 

answer 

(3) 


The  use  of  controls  was  intuitive. 

41.67% — 


- 34. /2% 

13.89% 

694% 

2.78%  Ll  1 

0.00% — — ■ - 

1(0)  2(2)  3(10)  4(25)  5(30)  No 

answer 

(5) 


The  choice  of  colors  in  user 
interface  made  the  instructions 
easy  to  read. 


50.00% 


f 

34  77% 

n 

4  17% 

0.00%  0.00%  g 

1(0)  2(0)  3(8)  4(36)  5(25)  No 

answer 

(3) 
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The  meaning  of  menu  items  was 
clear. 


44.44%“ 


I  was  able  to  complete  the  task 
(execute  the  mission)  very 
efficiently. 


1(3)  2(7)  3(16)  4(28)  5(16)  No 

answer 

(2) 


The  purpose  of  this  web-based 
simulation  was  clear. 


32r72%~ 


f 

29.17% 

f 

iy.44% 

11  1 1  0^ 

n 

±  ± ±  ±  /o 

1  390^ 

4.17% 

H 

1(1)  2(8)  3(14)  4(25)  5(21)  No 

answer 

(3) 


This  type  of  three-dimensional  (3D) 
web-based  simulations  could  be  a 
valuable  tool  for  training. 

47.22% 


1(1)  2(1)  3(7)  4(34)  5(27)  No 

answer 

(2) 
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This  web-based  simulation 
reminded  of  my  experience  in 
using  a  real  periscope. 


35:71%  35.7m 


answer 

(0) 


There  were  technical  difficulties, 
but  they  did  not  influence 
negatively  the  use  of  this  web- 
based  simulation. 


1(19)  2(10)  3(22)  4(14)  5(3)  No 

answer 

(4) 


I  did  not  experience  any  technical 
difficulties  with  this  web-based 
simulation. 


1(6)  2(3)  3(6)  4(18)  5(37)  No 

answer 

(2) 


This  web-based  simulation  would 
be  a  good  tool  for  training  of 
periscope  handling,  and  it  could  be 
used  to  help  improve  these  type  of 
skills. 


answer 

(0) 


125 


Hands-on  training  motivates  me. 


45.71%  44.29% 


answer 

(1) 


In  my  past  experience,  before  I 
would  start  training  with  a  full 
scale  simulator  I  felt  well  prepared 
to  take  part  in  that  training 
session. 


-41.67% 


- 30.56% 

f 

7 

f 

16.67% 

8.33% 

y 

0.00% 

1(1)  2(3)  3(11)  4(15)  5(6)  No 

answer 

(0) 


In  my  past  experience,  before  I 
would  start  training  with  a  full 
scale  simulator,  I  had  clear 
understanding  of  all  the  tasks  I  was 
expected  to  execute. 


1(0)  2(4)  3(9)  4(18)  5(5)  No 

answer 

(0) 


The  number  of  hours  I  had  in  a  full 
scale  simulator  was  enough  to 
understand  the  theory  and  master 
the  necessary  skills  I  needed  before 
going  to  the  related  operational 
environment. 


50m%- 


13.89% 


1(0)  2(4)  3(8)  4(18)  5(5)  No 

answer 

(1) 
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If  I  had  access  to  web-based 
simulation  that  I  could  use  from 
home,  I  would  use  it  to  improve  my 
performance. 


58.33% 


f 

11  110/^ 

19.44% 

2.^%  ^ 

p5.^% 

2.78% 

1(1)  2(4)  3(2)  4(21)  5(7)  No 

answer 

(1) 


Web-based  three  dimensional  (3D) 
simulation  is  a  valuable  tool  for 
training. 


If  I  had  a  choice,  I  would  replace 
some  hours  of  traditional 
classroom  lectures  for  web-based 
classes  that  I  could  access  from 
home. 


Have  you  ever  done  any  kind  of 
web-based  training  before? 


1(1)  2(10)  3(8)  4(33)  5(17) 


No 

answer 

(1) 


1  (2)  2  (2) 


3(5)  4(26)  5(34)  No 

answer 
(1) 


Regarding  web-based  (online) 
training,  I  would  prefer  doing  it: 


62.86% 


35.71% 

1A6% 

At  home  (44)  At  work  (25)  No  answer  (1) 
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If  some  web-based  training  is  found  equally  effective  on  all  those  devices, 
what  kind  of  device  would  you  prefer  to  use  for  your  web-based  training 

in  the  future? 


57.14% 


no 

0 

L2.86°/ 

0 

5  71% 

p — ^  1.43%  1.43% 

Desktop  computer  Laptop  computer  Tablet  (9)  Smartphone  (4)  Other  (1)  No  answer  (1) 

(15)  (40) 


When  do  you  think  a  web-based  simulation  would  be  more  useful  as  a 

training  tool? 


58.57% 


QC  ~710Z 

0.00% 

2.86%  i43o/„  143% 

c:2^] ^ 

Before  using  a  full  After  using  a  full  Both  before  and  I  do  not  think  it  Other  (1)  No  answer  (1) 

scale  simulator,  to  scale  simulator,  to  after  training  with  would  be  useful  at 
help  better  help  retain  a  full  scale  all.  (2) 

visualize  and  knowledge  and  simulator.  (41) 

understand  the  skills.  (0) 
theory.  (25) 
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